2025-11-26T14:16:49.913318Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7577037890415052958:2266];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:49.913338Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-26T14:16:49.923109Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7577037890661094112:2082];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:49.923246Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/isxu/0008ad/r3tmp/tmpF2RH46/pdisk_1.dat
2025-11-26T14:16:49.925094Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-26T14:16:49.925105Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-26T14:16:49.978781Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-26T14:16:49.978952Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-26T14:16:50.015673Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-26T14:16:50.015721Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-26T14:16:50.016875Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-26T14:16:50.019781Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-11-26T14:16:50.029465Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-26T14:16:50.029493Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-26T14:16:50.030641Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-26T14:16:50.030944Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 16536, node 1
2025-11-26T14:16:50.105675Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/isxu/0008ad/r3tmp/yandexdD53Og.tmp
2025-11-26T14:16:50.105693Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/isxu/0008ad/r3tmp/yandexdD53Og.tmp
2025-11-26T14:16:50.105767Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/isxu/0008ad/r3tmp/yandexdD53Og.tmp
2025-11-26T14:16:50.105825Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-26T14:16:50.172216Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-26T14:16:50.266853Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-26T14:16:50.300870Z INFO: TTestServer started on Port 5168 GrpcPort 16536
TClient is connected to server localhost:5168
PQClient connected to localhost:16536
WaitRootIsUp 'Root'...
TClient::Ls request: Root
2025-11-26T14:16:50.349337Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7577037894710020792:2316], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.349384Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.349609Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7577037894710020804:2319], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.349621Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7577037894710020805:2320], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.349642Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-26T14:16:50.381317Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-26T14:16:50.381731Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720658:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-11-26T14:16:50.403368Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7577037894710020808:2321], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720657 completed, doublechecking }
waiting...
2025-11-26T14:16:50.436587Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720659, at schemeshard: 72057594046644480
2025-11-26T14:16:50.441359Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720660:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
2025-11-26T14:16:50.482816Z node 1 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [1:7577037894710021001:2609] txid# 281474976720661, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 5], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
waiting...
waiting...
2025-11-26T14:16:50.693511Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7577037894956061723:2301], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-26T14:16:50.693840Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=ODkxNGMxNzItMWY5MTJhY2MtODc0NDQ0NTItZjg2MDI2Mzg=, ActorId: [2:7577037894956061675:2295], ActorState: ExecuteState, TraceId: 01kb08cbf3dy336j5av7feh5q9, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-26T14:16:50.694414Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-26T14:16:50.693192Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7577037894710021015:2328], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-26T14:16:50.697232Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=MjE3ZDE3YjctY2Y2M2JlODYtZDZlMmQ4YWYtZWNlOTgyYWM=, ActorId: [1:7577037894710020790:2315], ActorState: ExecuteState, TraceId: 01kb08cbd4abhht0b3qtmtehes, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-26T14:16:50.697354Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-26T14:16:50.736496Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-26T14:16:50.812769Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-26T14:16:50.878985Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720666:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
2025-11-26T14:16:50.914720Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-26T14:16:50.925042Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-26T14:16:51.050966Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720667. Ctx: { TraceId: 01kb08cbz86k3sg4618gwc6mex, Database: , SessionId: ydb://session/3?node_id=1&id=MTU3OGEwMGQtM2VkNWFhYjctNDU2YjBkZWUtYmFiNjc5Yzk=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7577037899004988891:3045] 
2025-11-26T14:16:54.914320Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7577037890415052958:2266];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:54.914370Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-26T14:16:54.924958Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7577037890661094112:2082];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:54.924990Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:16536
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:56.187231Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-26T14:16:56.227031Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7577037920479825755:3252] connected; active server actors: 1
2025-11-26T14:16:56.229966Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-26T14:16:56.230165Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:971: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-26T14:16:56.232572Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-26T14:16:56.247307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-26T14:16:56.261234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-26T14:16:56.261404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:785: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-26T14:16:56.261427Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:911: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-26T14:16:56.261442Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-26T14:16:56.264533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825796:2457], now have 1 active actors on pipe
2025-11-26T14:16:56.264551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825753:3251], now have 1 active actors on pipe
2025-11-26T14:16:56.265893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.264999Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-26T14:16:56.266448Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720674. Ctx: { TraceId: 01kb08ch513dxav71kx9ryrwg9, Database: , SessionId: ydb://session/3?node_id=1&id=YzJkYjdhNTYtOGVhZGI2NTktZmQ5YWY5NmUtN2U5NTg5N2E=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-11-26T14:16:56.269474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.276970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825858:3319], now have 1 active actors on pipe
2025-11-26T14:16:56.277420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--topic1 version0
2025-11-26T14:16:56.281037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.287006Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-26T14:16:56.287038Z node 2 :PERSQUEUE DEBUG: partition.cpp:1315: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-26T14:16:56.287221Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-26T14:16:56.287324Z node 2 :PERSQUEUE INFO: partition_init.cpp:1107: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7577037920725865838:2357]
2025-11-26T14:16:56.291119Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-26T14:16:56.291136Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-26T14:16:56.291143Z node 2 :PERSQUEUE INFO: partition.cpp:707: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7577037920725865838:2357]
2025-11-26T14:16:56.291151Z node 2 :PERSQUEUE DEBUG: partition.cpp:727: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-26T14:16:56.291176Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-26T14:16:56.291533Z node 2 :PERSQUEUE DEBUG: partition.cpp:4404: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-26T14:16:56.291540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.291551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2392: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-26T14:16:56.291625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.291628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.291631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.291633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.291650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.291651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.291653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.291655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.291657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.291666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-26T14:16:56.291675Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-26T14:16:56.291800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.291830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.293798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.293861Z node 2 :PERSQUEUE DEBUG: partition.cpp:1420: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764166616324, TxId 281474976720673
2025-11-26T14:16:56.293867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.293871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.293873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.293879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2456: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-26T14:16:56.293899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.293908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.293910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.293988Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.300935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.301005Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-26T14:16:56.301052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.301060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.301064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.301066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.301069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.301071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.301079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.301224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-26T14:16:56.301246Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:911: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-26T14:16:56.301381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.308654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.308701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.317405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.392483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.392503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.392507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.392511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.392513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.492970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.492989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.492992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.492996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.492999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.593116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.593136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.593139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.593147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.593150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.700346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.700371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.700375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.700379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.700382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1764166616324
    ParentPathId: 6
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1764166616324
    ParentPathId: 6
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:16536
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-26T14:16:56.800369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.800388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.800392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.800397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.800400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.801071Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:56.807394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825911:3359], now have 1 active actors on pipe
2025-11-26T14:16:56.806746Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.812491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.812526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.812576Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ebf47f8c-5501749a-72ae9f74-48c712d_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.812591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.812596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.812603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.812605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.812609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.812619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.812622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.812625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.812641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.812661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.813452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825911:3359] destroyed
2025-11-26T14:16:56.812864Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ebf47f8c-5501749a-72ae9f74-48c712d_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ebf47f8c-5501749a-72ae9f74-48c712d_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:56.834574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825914:3361], now have 1 active actors on pipe
2025-11-26T14:16:56.833484Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.838300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.838323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.838675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:56.839357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:56.839384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:56.839389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:56.839416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:56.839441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.839446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.839458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.839460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.839465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:56.839467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.839472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.839505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-26T14:16:56.839519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.839523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-26T14:16:56.839526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.839529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-26T14:16:56.840504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-26T14:16:56.840523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:56.840526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:56.840529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.840612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00002? size 1048780 WTime 1764166616840
2025-11-26T14:16:56.841997Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.842032Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 1048780
2025-11-26T14:16:56.854792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:56.854828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.854859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.854865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.854867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:56.854878Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-26T14:16:56.854882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.854887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 1, Offset: 0 is stored on disk
2025-11-26T14:16:56.854893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.854897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 2, Offset: 0 is stored on disk
2025-11-26T14:16:56.854953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-26T14:16:56.854960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-26T14:16:56.854965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.854969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.854971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.854973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.854975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.854985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.855003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.855046Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-26T14:16:56.855063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 1048780 count 1 last offset 0, current partition end offset: 1
2025-11-26T14:16:56.855067Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-26T14:16:56.855079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:56.855089Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-26T14:16:56.855105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-26T14:16:56.855302Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:56.855433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:56.855457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:56.855503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:56.855526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825914:3361] destroyed
2025-11-26T14:16:56.855533Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764166616836 queuesize 0 startOffset 0
2025-11-26T14:16:56.855109Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    PartNo: 0
    WriteTimestampMS: 1764166616836
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 16
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:56.858359Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.860422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825920:3364], now have 1 active actors on pipe
2025-11-26T14:16:56.860456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.860466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.860511Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b78c6ece-5541490a-1d89a8f7-e93f165b_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.860523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.860527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.860533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.860535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.860539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.860549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.860552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.860555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.860574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.860592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.860773Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.861178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825920:3364] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b78c6ece-5541490a-1d89a8f7-e93f165b_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b78c6ece-5541490a-1d89a8f7-e93f165b_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:56.876658Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.884329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825923:3365], now have 1 active actors on pipe
2025-11-26T14:16:56.884932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.884942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.885039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:56.885858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:56.885869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:56.885874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:56.885904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:56.885926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.885930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.885940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.885943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.885947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:56.885949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.885954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.885990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-26T14:16:56.886003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.886006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-26T14:16:56.886009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.886011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-26T14:16:56.886962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 1048790 count 1 nextOffset 2 batches 3
2025-11-26T14:16:56.886974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:56.886978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:56.886982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.887047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00002? size 1048780 WTime 1764166616884
2025-11-26T14:16:56.887300Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.887313Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 1048780
2025-11-26T14:16:56.904379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:56.904601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:56.904619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.904645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.904654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.904666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-26T14:16:56.904670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.904673Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 1, Offset: 1 is stored on disk
2025-11-26T14:16:56.904679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.904683Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 2, Offset: 1 is stored on disk
2025-11-26T14:16:56.904721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.904725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.904727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.904730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.904732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.904743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.904759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.904966Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.904885Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:56.905335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825923:3365] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    PartNo: 0
    WriteTimestampMS: 1764166616884
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 20
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:56.909278Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.912864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825929:3368], now have 1 active actors on pipe
2025-11-26T14:16:56.913068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.913078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.913109Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b0ec95ab-9d0f7a0c-551ca671-4279d744_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.913120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.913125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.913130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.913132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.913136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.913146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.913148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.913150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.913168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.913184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.913311Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.913635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825929:3368] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b0ec95ab-9d0f7a0c-551ca671-4279d744_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b0ec95ab-9d0f7a0c-551ca671-4279d744_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:56.927026Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.930497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825934:3371], now have 1 active actors on pipe
2025-11-26T14:16:56.937362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.937383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.937485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:56.938284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:56.938294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:56.938299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:56.938322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:56.938342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.938346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.938358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.938361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.938366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:56.938368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.938372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.938406Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-26T14:16:56.938420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.938422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-26T14:16:56.938425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.938428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-26T14:16:56.939582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 1048790 count 1 nextOffset 3 batches 3
2025-11-26T14:16:56.939593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:56.939596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:56.939599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.939665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00002? size 1048780 WTime 1764166616936
2025-11-26T14:16:56.941075Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.941096Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 1048780
2025-11-26T14:16:56.977970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:56.978006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.978034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.978041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.978054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-26T14:16:56.978058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.978063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 1, Offset: 2 is stored on disk
2025-11-26T14:16:56.978069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.978073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 2, Offset: 2 is stored on disk
2025-11-26T14:16:56.978121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.978126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.978128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.978132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.978135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.978144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.978167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    PartNo: 0
    WriteTimestampMS: 1764166616936
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 41
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:56.978334Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:56.980295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825934:3371] destroyed
2025-11-26T14:16:56.980531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825948:3379], now have 1 active actors on pipe
2025-11-26T14:16:56.980703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.980712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.980746Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a90bcf2-4a27d680-c8dd922-a6282175_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.980756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.980760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.980766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.980768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.980772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.980781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.980784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.980787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.980801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.980815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a90bcf2-4a27d680-c8dd922-a6282175_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:56.978451Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.980002Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.980967Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a90bcf2-4a27d680-c8dd922-a6282175_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:56.989481Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.989604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825948:3379] destroyed
2025-11-26T14:16:56.996872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037920479825951:3380], now have 1 active actors on pipe
2025-11-26T14:16:56.998379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.998406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.998477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:56.998525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:56.998541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:56.998547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:56.998570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:56.998602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.998606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.998615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.998617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.998622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:56.998624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.998629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.998661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-26T14:16:56.998671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.998675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-26T14:16:56.998679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.998682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-26T14:16:56.998977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 1048790 count 1 nextOffset 4 batches 3
2025-11-26T14:16:56.998988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:56.998991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:56.998997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.999078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00002? size 1048780 WTime 1764166616998
2025-11-26T14:16:56.999259Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.999275Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.012558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.012942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.012960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.012985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.012993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.013006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-26T14:16:57.013010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.013015Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 1, Offset: 3 is stored on disk
2025-11-26T14:16:57.013021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.013026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 2, Offset: 3 is stored on disk
2025-11-26T14:16:57.013072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.013076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.013079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.013086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.013090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.013100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.013117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.013177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.013354Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    PartNo: 0
    WriteTimestampMS: 1764166616998
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.014490Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.015277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037920479825951:3380] destroyed
2025-11-26T14:16:57.015291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793252:3383], now have 1 active actors on pipe
2025-11-26T14:16:57.015302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.015308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.015336Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cdd6f500-a2b5f1aa-89c6a5f2-d32efc71_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.015344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.015346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.015350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.015352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.015356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.015363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.015366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.015369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.015380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.015391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.015537Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cdd6f500-a2b5f1aa-89c6a5f2-d32efc71_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cdd6f500-a2b5f1aa-89c6a5f2-d32efc71_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.024076Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.025843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793252:3383] destroyed
2025-11-26T14:16:57.025865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793255:3384], now have 1 active actors on pipe
2025-11-26T14:16:57.026192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.026201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.026303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.026761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.026776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.026781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.026803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.026822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.026825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.026833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.026836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.026840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.026842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.026847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.026875Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-26T14:16:57.026885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.026889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-26T14:16:57.026892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.026896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-26T14:16:57.027323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 1048790 count 1 nextOffset 5 batches 3
2025-11-26T14:16:57.027335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.027338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.027342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.027422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00002? size 1048780 WTime 1764166617026
2025-11-26T14:16:57.027605Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.027620Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.033913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.033940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.033967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.033973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.033983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-26T14:16:57.033987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.033991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 1, Offset: 4 is stored on disk
2025-11-26T14:16:57.033996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.034000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 2, Offset: 4 is stored on disk
2025-11-26T14:16:57.034039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.034043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.034045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.034049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.034051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.034061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.034077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.034225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.034307Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.034824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793255:3384] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    PartNo: 0
    WriteTimestampMS: 1764166617026
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.039517Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.040102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793260:3387], now have 1 active actors on pipe
2025-11-26T14:16:57.040279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.040289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.040316Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|27a6dd73-2fceb017-73153d96-8e4bb9e9_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.040325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.040329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.040333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.040335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.040338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.040346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.040348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.040352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.040365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.040381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.040507Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.040945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793260:3387] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|27a6dd73-2fceb017-73153d96-8e4bb9e9_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|27a6dd73-2fceb017-73153d96-8e4bb9e9_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.048322Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.050122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793263:3388], now have 1 active actors on pipe
2025-11-26T14:16:57.051457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.051471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.051541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.051590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.051600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.051604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.051618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.051635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.051639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.051647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.051649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.051653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.051655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.051659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.051683Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-26T14:16:57.051696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.051699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-26T14:16:57.051703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.051707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-26T14:16:57.052559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 1048790 count 1 nextOffset 6 batches 3
2025-11-26T14:16:57.052571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.052575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.052578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.052636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00002? size 1048780 WTime 1764166617048
2025-11-26T14:16:57.052943Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.052955Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.075270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.075293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.075315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.075321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.075330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-26T14:16:57.075334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.075339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 1, Offset: 5 is stored on disk
2025-11-26T14:16:57.075344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.075347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 2, Offset: 5 is stored on disk
2025-11-26T14:16:57.075384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.075386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.075390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.075394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.075395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.075403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.075414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.075548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.075614Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.076036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793263:3388] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    PartNo: 0
    WriteTimestampMS: 1764166617048
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 23
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.077350Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.077844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793271:3391], now have 1 active actors on pipe
2025-11-26T14:16:57.078004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.078010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.078031Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|621f6866-27384bf2-291e132c-519418e2_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.078037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.078039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.078043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.078045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.078048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.078054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.078056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.078058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.078067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.078076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.078199Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|621f6866-27384bf2-291e132c-519418e2_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|621f6866-27384bf2-291e132c-519418e2_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.083359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793271:3391] destroyed
2025-11-26T14:16:57.091866Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.092579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793275:3393], now have 1 active actors on pipe
2025-11-26T14:16:57.093840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.093856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.093930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.094018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.094029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.094034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.094052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.094071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.094075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.094085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.094087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.094091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.094093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.094098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.094124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-26T14:16:57.094135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.094141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-26T14:16:57.094144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.094146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-26T14:16:57.095563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 1048790 count 1 nextOffset 7 batches 3
2025-11-26T14:16:57.095580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.095583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.095586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.095650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00002? size 1048780 WTime 1764166617092
2025-11-26T14:16:57.095859Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.095872Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.103389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.103420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.103445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.103451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.103462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-26T14:16:57.103465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.103469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 1, Offset: 6 is stored on disk
2025-11-26T14:16:57.103474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.103478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 2, Offset: 6 is stored on disk
2025-11-26T14:16:57.103518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.103524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.103526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.103529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.103531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.103541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.103556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.103612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.104273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793275:3393] destroyed
2025-11-26T14:16:57.103682Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    PartNo: 0
    WriteTimestampMS: 1764166617092
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.105161Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.108148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793280:3396], now have 1 active actors on pipe
2025-11-26T14:16:57.108187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.108194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.108235Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|89f85acc-556d49fe-6b3872ec-53932a7_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.108245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.108248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.108254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.108256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.108272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.108282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.108286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.108292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.108309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.108325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.112388Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|89f85acc-556d49fe-6b3872ec-53932a7_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:57.115156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.115179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.115182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.115186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.115188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.115215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793280:3396] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|89f85acc-556d49fe-6b3872ec-53932a7_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.126097Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.128518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793284:3398], now have 1 active actors on pipe
2025-11-26T14:16:57.130565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.130598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.130681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.130729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.130740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.130746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.130770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.130792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.130795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.130804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.130806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.130811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.130813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.130818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.130846Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-26T14:16:57.130858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.130864Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-26T14:16:57.130867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.130870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-26T14:16:57.131444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 2 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 1048790 count 1 nextOffset 8 batches 3
2025-11-26T14:16:57.131456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.131458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.131462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.131537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00002? size 1048780 WTime 1764166617128
2025-11-26T14:16:57.131704Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.131718Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.135327Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.135346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.135370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.135375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.135384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-26T14:16:57.135388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.135393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 1, Offset: 7 is stored on disk
2025-11-26T14:16:57.135398Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.135403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 2, Offset: 7 is stored on disk
2025-11-26T14:16:57.135437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.135441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.135443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.135445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.135447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.135457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.135481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-26T14:16:57.135497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.135511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-26T14:16:57.135514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-26T14:16:57.135526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-26T14:16:57.135542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-26T14:16:57.135562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-26T14:16:57.135572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-26T14:16:57.135604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-26T14:16:57.135614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-26T14:16:57.135624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-26T14:16:57.135635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-26T14:16:57.135646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 1 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:57.135650Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:57.135655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:57.135659Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.135663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.135667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.135671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.135674Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.135681Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-26T14:16:57.135732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-26T14:16:57.135741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-26T14:16:57.135743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.135747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00002?, parameters.CurOffset=0
2025-11-26T14:16:57.135897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/3
2025-11-26T14:16:57.135904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-26T14:16:57.135911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-26T14:16:57.136080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.1/3
2025-11-26T14:16:57.136085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-26T14:16:57.136088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:1 appended
2025-11-26T14:16:57.136103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.2/3
2025-11-26T14:16:57.136107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-26T14:16:57.136365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-26T14:16:57.136372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:2 appended
2025-11-26T14:16:57.136376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-26T14:16:57.136380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.136383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00002?, parameters.CurOffset=1
2025-11-26T14:16:57.136515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/3
2025-11-26T14:16:57.136520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-26T14:16:57.136522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-26T14:16:57.136835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.1/3
2025-11-26T14:16:57.136839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-26T14:16:57.136843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:1 appended
2025-11-26T14:16:57.136871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.2/3
2025-11-26T14:16:57.136874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-26T14:16:57.137085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097545 count 2 nextOffset 2 batches 5
2025-11-26T14:16:57.137091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:2 appended
2025-11-26T14:16:57.137095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-26T14:16:57.137098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.137100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00002?, parameters.CurOffset=2
2025-11-26T14:16:57.137240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/3
2025-11-26T14:16:57.137245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-26T14:16:57.137247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-26T14:16:57.137755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.1/3
2025-11-26T14:16:57.137760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-26T14:16:57.137763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:1 appended
2025-11-26T14:16:57.137777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.2/3
2025-11-26T14:16:57.137780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-26T14:16:57.138348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146300 count 3 nextOffset 3 batches 7
2025-11-26T14:16:57.138354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:2 appended
2025-11-26T14:16:57.138357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-26T14:16:57.138359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.138361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00002?, parameters.CurOffset=3
2025-11-26T14:16:57.138501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/3
2025-11-26T14:16:57.138505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-26T14:16:57.138507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-26T14:16:57.138658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.1/3
2025-11-26T14:16:57.138662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-26T14:16:57.138664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:1 appended
2025-11-26T14:16:57.138679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.2/3
2025-11-26T14:16:57.138681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-26T14:16:57.139202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195055 count 4 nextOffset 4 batches 9
2025-11-26T14:16:57.139207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:2 appended
2025-11-26T14:16:57.139211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-26T14:16:57.139212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.139215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00002?, parameters.CurOffset=4
2025-11-26T14:16:57.139364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/3
2025-11-26T14:16:57.139371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-26T14:16:57.139373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-26T14:16:57.139855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.1/3
2025-11-26T14:16:57.139860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-26T14:16:57.139862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:1 appended
2025-11-26T14:16:57.139874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.2/3
2025-11-26T14:16:57.139877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-26T14:16:57.140854Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.140707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.140725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.140729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.140731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.140735Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.140738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.140742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.140745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.140747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63'
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    PartNo: 0
    WriteTimestampMS: 1764166617128
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.148360Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.149566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243810 count 5 nextOffset 5 batches 11
2025-11-26T14:16:57.149591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:2 appended
2025-11-26T14:16:57.149600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-26T14:16:57.149603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.149607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00002?, parameters.CurOffset=5
2025-11-26T14:16:57.150079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/3
2025-11-26T14:16:57.150088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-26T14:16:57.150092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-26T14:16:57.150546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.1/3
2025-11-26T14:16:57.150552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-26T14:16:57.150555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:1 appended
2025-11-26T14:16:57.150572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.2/3
2025-11-26T14:16:57.150575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-26T14:16:57.150774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292565 count 6 nextOffset 6 batches 13
2025-11-26T14:16:57.150784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:2 appended
2025-11-26T14:16:57.150788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-26T14:16:57.150790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.150793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00002?, parameters.CurOffset=6
2025-11-26T14:16:57.150950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/3
2025-11-26T14:16:57.150958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-26T14:16:57.150961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-26T14:16:57.151117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.1/3
2025-11-26T14:16:57.151120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-26T14:16:57.151123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:1 appended
2025-11-26T14:16:57.151137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.2/3
2025-11-26T14:16:57.151139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-26T14:16:57.151355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341320 count 7 nextOffset 7 batches 15
2025-11-26T14:16:57.151360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:2 appended
2025-11-26T14:16:57.151364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-26T14:16:57.151366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.151369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00002?, parameters.CurOffset=7
2025-11-26T14:16:57.151532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/3
2025-11-26T14:16:57.151536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-26T14:16:57.151538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-26T14:16:57.151706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.1/3
2025-11-26T14:16:57.151710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-26T14:16:57.151712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:1 appended
2025-11-26T14:16:57.151724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.2/3
2025-11-26T14:16:57.151726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-26T14:16:57.172840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 7 partNo 2 result is x0000000000_00000000000000000000_00000_0000000007_00016 size 8365339
2025-11-26T14:16:57.172870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000007_00016 new key d0000000000_00000000000000000000_00000_0000000007_00016 size 8365339 WTime 1764166617171
2025-11-26T14:16:57.172919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 2 FormedBlobsCount 1 NewHead: Offset 7 PartNo 2 PackedSize 24736 count 1 nextOffset 8 batches 1
2025-11-26T14:16:57.172928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:2 appended
2025-11-26T14:16:57.172998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 0 endOffset 0 curOffset 8 d0000000000_00000000000000000007_00002_0000000001_00000| size 24726 WTime 1764166617171
2025-11-26T14:16:57.173051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793284:3398] destroyed
2025-11-26T14:16:57.173062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793290:3402], now have 1 active actors on pipe
2025-11-26T14:16:57.173087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.173093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.173109Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.173123Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 7 size 8365339
2025-11-26T14:16:57.173130Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 1 size 24726
2025-11-26T14:16:57.173162Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|73ac03db-382956bb-d08d6c87-21f0ee4f_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.173171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.173175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.173183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.173185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.173189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.173198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.173201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.173203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.173216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.173272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.175544Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|73ac03db-382956bb-d08d6c87-21f0ee4f_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:57.176415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793290:3402] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|73ac03db-382956bb-d08d6c87-21f0ee4f_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.184440Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.192323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793308:3411], now have 1 active actors on pipe
2025-11-26T14:16:57.192884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.192895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.193002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.193064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.193076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.193081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.193107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.193130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.193134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.193143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.193146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.193154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.193157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.193161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.193194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-26T14:16:57.193207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.193210Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-26T14:16:57.193214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.193217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-26T14:16:57.193457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-26T14:16:57.193469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.193473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.193476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.193550Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00002? size 1048780 WTime 1764166617193
2025-11-26T14:16:57.193797Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.193811Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.216372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.219785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.219812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 24726 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.219831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.219879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.219886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.219981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-26T14:16:57.219991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-26T14:16:57.229990Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.228720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.228753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.228780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.228787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.228799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-26T14:16:57.228802Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.228807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 1, Offset: 8 is stored on disk
2025-11-26T14:16:57.228812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.228816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 2, Offset: 8 is stored on disk
2025-11-26T14:16:57.228862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.228865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.228867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.228870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.228872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.228916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.228934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.228951Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.228958Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00002?(+) to d0000000000_00000000000000000000_00000_0000000001_00002?(+)
2025-11-26T14:16:57.228965Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00002?(+) to d0000000000_00000000000000000001_00000_0000000001_00002?(+)
2025-11-26T14:16:57.228967Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00002?(+) to d0000000000_00000000000000000002_00000_0000000001_00002?(+)
2025-11-26T14:16:57.228970Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00002?(+) to d0000000000_00000000000000000003_00000_0000000001_00002?(+)
2025-11-26T14:16:57.228973Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00002?(+) to d0000000000_00000000000000000004_00000_0000000001_00002?(+)
2025-11-26T14:16:57.228976Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00002?(+) to d0000000000_00000000000000000005_00000_0000000001_00002?(+)
2025-11-26T14:16:57.228978Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00002?(+) to d0000000000_00000000000000000006_00000_0000000001_00002?(+)
2025-11-26T14:16:57.228980Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00002?(+) to d0000000000_00000000000000000007_00000_0000000001_00002?(+)
2025-11-26T14:16:57.229236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    PartNo: 0
    WriteTimestampMS: 1764166617192
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 34
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.236230Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.235291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793308:3411] destroyed
2025-11-26T14:16:57.235414Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.235471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.235489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.235514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.235518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.235522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.235525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.235527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.235537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.235604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.235615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.235621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.235626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.235638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.235644Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.235649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.235655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.239504Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.238424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793321:3417], now have 1 active actors on pipe
2025-11-26T14:16:57.239210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.239220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.239257Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|61db7563-54de00c-a160da3f-71aa623e_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.239268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.239273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.239279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.239282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.239286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.239295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.239298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.239302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.239318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.239335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.240120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793321:3417] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|61db7563-54de00c-a160da3f-71aa623e_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|61db7563-54de00c-a160da3f-71aa623e_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.248927Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.252912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793339:3425], now have 1 active actors on pipe
2025-11-26T14:16:57.254069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.254083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.254183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.254249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.254259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.254265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.254280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.254301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.254305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.254315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.254317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.254322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.254325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.254329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.254359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-26T14:16:57.254372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.254375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-26T14:16:57.254378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.254380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-26T14:16:57.254768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 1048790 count 1 nextOffset 10 batches 3
2025-11-26T14:16:57.254779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.254782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.254785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.254847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00002? size 1048780 WTime 1764166617241
2025-11-26T14:16:57.255017Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.255030Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.272248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.272322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.272343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.272348Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.272355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.272366Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-26T14:16:57.272369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.272375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 1, Offset: 9 is stored on disk
2025-11-26T14:16:57.272380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.272383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 2, Offset: 9 is stored on disk
2025-11-26T14:16:57.272423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.272430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.272432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.272435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.272437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.272446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.272463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.272926Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    PartNo: 0
    WriteTimestampMS: 1764166617241
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 30
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.274155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793339:3425] destroyed
2025-11-26T14:16:57.274244Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.276304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793345:3429], now have 1 active actors on pipe
2025-11-26T14:16:57.276509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.276519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.276551Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8be0fabf-2dab6841-c63e3cd7-8b3370cb_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.276565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.276571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.276576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.276579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.276583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.276593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.276595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.276598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.276613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.276628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.276977Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8be0fabf-2dab6841-c63e3cd7-8b3370cb_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:57.277413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793345:3429] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8be0fabf-2dab6841-c63e3cd7-8b3370cb_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.284708Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.285446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793348:3430], now have 1 active actors on pipe
2025-11-26T14:16:57.286900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.286922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.287027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.287100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.287109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.287113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.287131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.287150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.287169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.287179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.287182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.287186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.287188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.287194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.287224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-26T14:16:57.287235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.287238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-26T14:16:57.287242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.287245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-26T14:16:57.287577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 1048790 count 1 nextOffset 11 batches 3
2025-11-26T14:16:57.287590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.287593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.287597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.287661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00002? size 1048780 WTime 1764166617285
2025-11-26T14:16:57.287928Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.287949Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.291234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.291252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.291272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.291280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.291290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-26T14:16:57.291293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.291297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 1, Offset: 10 is stored on disk
2025-11-26T14:16:57.291302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.291305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 2, Offset: 10 is stored on disk
2025-11-26T14:16:57.291333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.291335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.291337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.291339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.291341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.291348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.291362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.291493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.291571Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    PartNo: 0
    WriteTimestampMS: 1764166617285
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.292850Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.293779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793348:3430] destroyed
2025-11-26T14:16:57.293797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793353:3433], now have 1 active actors on pipe
2025-11-26T14:16:57.293806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.293811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.293840Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a5eabf34-4a57fb5c-4bf7780d-d0bc271c_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.293848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.293853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.293857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.293860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.293863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.293870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.293873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.293875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.293886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.293897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.294038Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.294452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793353:3433] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a5eabf34-4a57fb5c-4bf7780d-d0bc271c_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a5eabf34-4a57fb5c-4bf7780d-d0bc271c_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.301324Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.302004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793356:3434], now have 1 active actors on pipe
2025-11-26T14:16:57.303270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.303288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.303353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.303588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.303597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.303603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.303617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.303651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.303656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.303664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.303666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.303670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.303672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.303676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.303703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-26T14:16:57.303714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.303717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-26T14:16:57.303720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.303724Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-26T14:16:57.303973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 1048790 count 1 nextOffset 12 batches 3
2025-11-26T14:16:57.303981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.303984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.303987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.304055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00002? size 1048780 WTime 1764166617303
2025-11-26T14:16:57.304202Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.304214Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.307485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.307505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.307524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.307531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.307540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-26T14:16:57.307543Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.307546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 1, Offset: 11 is stored on disk
2025-11-26T14:16:57.307551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.307555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 2, Offset: 11 is stored on disk
2025-11-26T14:16:57.307586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.307589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.307591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.307594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.307597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.307604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.307617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.307733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.307826Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    PartNo: 0
    WriteTimestampMS: 1764166617303
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.309235Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.309808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793356:3434] destroyed
2025-11-26T14:16:57.309822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793361:3437], now have 1 active actors on pipe
2025-11-26T14:16:57.310006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.310014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.310037Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|eff57ace-d34f485f-8dc86238-b2ce13b6_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.310046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.310049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.310053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.310055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.310058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.310065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.310068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.310070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.310081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.310093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.310213Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|eff57ace-d34f485f-8dc86238-b2ce13b6_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|eff57ace-d34f485f-8dc86238-b2ce13b6_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.313740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793361:3437] destroyed
2025-11-26T14:16:57.317493Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.319176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.319190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.319193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.319195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.319197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.319219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793367:3439], now have 1 active actors on pipe
2025-11-26T14:16:57.319501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.319508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.319606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.319669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.319677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.319682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.319694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.319707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.319710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.319717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.319719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.319722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.319723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.319727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.319753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-26T14:16:57.319764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.319766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-26T14:16:57.319770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.319772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-26T14:16:57.320026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 1048790 count 1 nextOffset 13 batches 3
2025-11-26T14:16:57.320034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.320037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.320039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.320096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00002? size 1048780 WTime 1764166617320
2025-11-26T14:16:57.320275Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.320289Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.323537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.323558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.323601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.328592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.328630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.328637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.328649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-26T14:16:57.328652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.328657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 1, Offset: 12 is stored on disk
2025-11-26T14:16:57.328663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.328668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 2, Offset: 12 is stored on disk
2025-11-26T14:16:57.328710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.328713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.328715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.328718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.328721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.328731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.328754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.328965Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.329523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793367:3439] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    PartNo: 0
    WriteTimestampMS: 1764166617319
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.336230Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.344171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793380:3443], now have 1 active actors on pipe
2025-11-26T14:16:57.344197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.344204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.344244Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b95e403d-69876c79-b4d2edf5-ad312bd_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.344255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.344283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.344290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.344292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.344296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.344306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.344309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.344314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.344329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.344344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.344464Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.345274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793380:3443] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b95e403d-69876c79-b4d2edf5-ad312bd_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b95e403d-69876c79-b4d2edf5-ad312bd_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.356637Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.358120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793383:3446], now have 1 active actors on pipe
2025-11-26T14:16:57.358628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.358649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.358754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.358805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.358816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.358821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.358837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.358857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.358861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.358870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.358873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.358878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.358881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.358886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.358915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-26T14:16:57.358928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.358932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-26T14:16:57.358936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.358939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-26T14:16:57.359192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 1048790 count 1 nextOffset 14 batches 3
2025-11-26T14:16:57.359211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.359215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.359218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.359293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00002? size 1048780 WTime 1764166617358
2025-11-26T14:16:57.359438Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.359452Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.363067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.363093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.363117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.363127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.363137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-26T14:16:57.363139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.363141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.363145Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 1, Offset: 13 is stored on disk
2025-11-26T14:16:57.363151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.363157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 2, Offset: 13 is stored on disk
2025-11-26T14:16:57.363192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.363196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.363197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.363200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.363202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.363211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.363240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.363496Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    PartNo: 0
    WriteTimestampMS: 1764166617358
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.363979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793383:3446] destroyed
2025-11-26T14:16:57.364601Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.365827Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8c0ac9c8-72540ce9-29cbbda4-81970b86_14"
    Status: Active
    SeqNo: 0
  }
}
2025-11-26T14:16:57.365623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793388:3449], now have 1 active actors on pipe
2025-11-26T14:16:57.365644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
CallPersQueueGRPC request to localhost:165362025-11-26T14:16:57.365651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.365679Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8c0ac9c8-72540ce9-29cbbda4-81970b86_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.365687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.365691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.365695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.365698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.365701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.365709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.365712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.365715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.365726Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.365737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.366255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793388:3449] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8c0ac9c8-72540ce9-29cbbda4-81970b86_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.373527Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.374119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793391:3450], now have 1 active actors on pipe
2025-11-26T14:16:57.375311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.375332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.375438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.375504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.375521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.375526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.375548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.375569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.375573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.375581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.375584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.375589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.375591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.375597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.375627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-26T14:16:57.375641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.375644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-26T14:16:57.375648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.375651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-26T14:16:57.375910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 1048790 count 1 nextOffset 15 batches 3
2025-11-26T14:16:57.375921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.375924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.375928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.376001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00002? size 1048780 WTime 1764166617375
2025-11-26T14:16:57.376149Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.376165Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.382315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.382342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.382364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.382368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.382376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-26T14:16:57.382378Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.382382Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 1, Offset: 14 is stored on disk
2025-11-26T14:16:57.382386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.382393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 2, Offset: 14 is stored on disk
2025-11-26T14:16:57.382425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.382427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.382428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.382429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.382431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.382438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.382450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.382501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.382867Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.383275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793391:3450] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    PartNo: 0
    WriteTimestampMS: 1764166617375
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.384184Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.384721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793396:3453], now have 1 active actors on pipe
2025-11-26T14:16:57.385021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.385031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.385062Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|23014306-958b0376-393ace0c-426f8f96_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.385072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.385076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.385081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.385083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.385087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.385096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.385100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.385103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.385117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.385130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.385258Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|23014306-958b0376-393ace0c-426f8f96_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:57.388069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793396:3453] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|23014306-958b0376-393ace0c-426f8f96_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.394051Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.394639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793399:3454], now have 1 active actors on pipe
2025-11-26T14:16:57.395828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.395851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.395924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.396019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.396029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.396035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.396055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.396075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.396080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.396091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.396095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.396099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.396104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.396108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.396140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-26T14:16:57.396151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.396154Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-26T14:16:57.396158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.396161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-26T14:16:57.396438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 2 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 1048790 count 1 nextOffset 16 batches 3
2025-11-26T14:16:57.396448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.396452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.396456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.396535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00002? size 1048780 WTime 1764166617395
2025-11-26T14:16:57.396696Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.396710Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.404340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.404372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.404399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.404405Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.404416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-26T14:16:57.404419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.404425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 1, Offset: 15 is stored on disk
2025-11-26T14:16:57.404432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.404438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 2, Offset: 15 is stored on disk
2025-11-26T14:16:57.404479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.404482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.404484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.404487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.404489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.404500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.404522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-26T14:16:57.404543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.404557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-26T14:16:57.404560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-26T14:16:57.404572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-26T14:16:57.404582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-26T14:16:57.404595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-26T14:16:57.404604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-26T14:16:57.404623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-26T14:16:57.404632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-26T14:16:57.404643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-26T14:16:57.404653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-26T14:16:57.404665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.404698Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-26T14:16:57.404728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-26T14:16:57.404737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-26T14:16:57.404740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.404744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00002?, parameters.CurOffset=8
2025-11-26T14:16:57.404914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/3
2025-11-26T14:16:57.404923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-26T14:16:57.404928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-26T14:16:57.405105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.1/3
2025-11-26T14:16:57.405109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-26T14:16:57.405113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:1 appended
2025-11-26T14:16:57.405132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.2/3
2025-11-26T14:16:57.405134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-26T14:16:57.405369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-26T14:16:57.405375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:2 appended
2025-11-26T14:16:57.405379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-26T14:16:57.405382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.405385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00002?, parameters.CurOffset=9
2025-11-26T14:16:57.405523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/3
2025-11-26T14:16:57.405528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-26T14:16:57.405531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-26T14:16:57.405668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.1/3
2025-11-26T14:16:57.405673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-26T14:16:57.405676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:1 appended
2025-11-26T14:16:57.405690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.2/3
2025-11-26T14:16:57.405693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-26T14:16:57.406078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 2097545 count 2 nextOffset 10 batches 5
2025-11-26T14:16:57.406083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:2 appended
2025-11-26T14:16:57.406087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-26T14:16:57.406089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.406092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00002?, parameters.CurOffset=10
2025-11-26T14:16:57.406225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/3
2025-11-26T14:16:57.406229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-26T14:16:57.406232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-26T14:16:57.406379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.1/3
2025-11-26T14:16:57.406383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-26T14:16:57.406385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:1 appended
2025-11-26T14:16:57.406398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.2/3
2025-11-26T14:16:57.406402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-26T14:16:57.406575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 3146300 count 3 nextOffset 11 batches 7
2025-11-26T14:16:57.406594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:2 appended
2025-11-26T14:16:57.406598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-26T14:16:57.406601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.406604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00002?, parameters.CurOffset=11
2025-11-26T14:16:57.406737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/3
2025-11-26T14:16:57.406742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-26T14:16:57.406744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-26T14:16:57.406880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.1/3
2025-11-26T14:16:57.406884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-26T14:16:57.406887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:1 appended
2025-11-26T14:16:57.406897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.2/3
2025-11-26T14:16:57.406904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-26T14:16:57.407072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 4195055 count 4 nextOffset 12 batches 9
2025-11-26T14:16:57.407076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:2 appended
2025-11-26T14:16:57.407079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-26T14:16:57.407082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.407085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00002?, parameters.CurOffset=12
2025-11-26T14:16:57.407249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/3
2025-11-26T14:16:57.407254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-26T14:16:57.407257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-26T14:16:57.407404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.1/3
2025-11-26T14:16:57.407407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-26T14:16:57.407410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:1 appended
2025-11-26T14:16:57.407421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.2/3
2025-11-26T14:16:57.407423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-26T14:16:57.407603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 5243810 count 5 nextOffset 13 batches 11
2025-11-26T14:16:57.407608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:2 appended
2025-11-26T14:16:57.407612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-26T14:16:57.407614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.407617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00002?, parameters.CurOffset=13
2025-11-26T14:16:57.407748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/3
2025-11-26T14:16:57.407752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-26T14:16:57.407755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-26T14:16:57.407893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.1/3
2025-11-26T14:16:57.407896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-26T14:16:57.407900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:1 appended
2025-11-26T14:16:57.407910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.2/3
2025-11-26T14:16:57.407913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-26T14:16:57.409039Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    PartNo: 0
    WriteTimestampMS: 1764166617395
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.408890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.408907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.408910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.408914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.408919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.408923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.408927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.408931Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.408934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.409841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 6292565 count 6 nextOffset 14 batches 13
2025-11-26T14:16:57.409852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:2 appended
2025-11-26T14:16:57.409858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-26T14:16:57.409861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.409864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00002?, parameters.CurOffset=14
2025-11-26T14:16:57.410005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/3
2025-11-26T14:16:57.410011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-26T14:16:57.410015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-26T14:16:57.410161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.1/3
2025-11-26T14:16:57.410166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-26T14:16:57.410169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:1 appended
2025-11-26T14:16:57.410180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.2/3
2025-11-26T14:16:57.410183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-26T14:16:57.410387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 7341320 count 7 nextOffset 15 batches 15
2025-11-26T14:16:57.410392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:2 appended
2025-11-26T14:16:57.410396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-26T14:16:57.410398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.410402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00002?, parameters.CurOffset=15
2025-11-26T14:16:57.410468Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.410568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/3
2025-11-26T14:16:57.410595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-26T14:16:57.410597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-26T14:16:57.410745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.1/3
2025-11-26T14:16:57.410748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-26T14:16:57.418419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 15 partNo 1 result is x0000000000_00000000000000000007_00002_0000000008_00015 size 7878063
2025-11-26T14:16:57.418443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:1 appended
2025-11-26T14:16:57.418467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.2/3
2025-11-26T14:16:57.418472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-26T14:16:57.418484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000007_00002_0000000008_00015 new key d0000000000_00000000000000000007_00002_0000000008_00015 size 7878063 WTime 1764166617418
2025-11-26T14:16:57.418633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 2 FormedBlobsCount 1 NewHead: Offset 15 PartNo 1 PackedSize 536763 count 1 nextOffset 16 batches 2
2025-11-26T14:16:57.418640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:2 appended
2025-11-26T14:16:57.418691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 7 endOffset 8 curOffset 16 d0000000000_00000000000000000015_00001_0000000001_00001| size 536753 WTime 1764166617418
2025-11-26T14:16:57.418818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793399:3454] destroyed
2025-11-26T14:16:57.418827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793405:3458], now have 1 active actors on pipe
2025-11-26T14:16:57.418850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.418857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.418871Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.418883Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 8 size 7878063
2025-11-26T14:16:57.418890Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 1 size 536753
2025-11-26T14:16:57.418924Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1f789be0-47699c7b-813ab9ad-51457178_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.418937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.418941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.418947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.418949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.418953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.418962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.418966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.418969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.418984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.419036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.419436Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1f789be0-47699c7b-813ab9ad-51457178_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1f789be0-47699c7b-813ab9ad-51457178_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.424320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.424339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.424341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.424344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.424347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.424372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793405:3458] destroyed
2025-11-26T14:16:57.427458Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.430388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793412:3461], now have 1 active actors on pipe
2025-11-26T14:16:57.437466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.437502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.437586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.437640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.437653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.437659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.437682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.437702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.437706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.437715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.437717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.437722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.437724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.437730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.437761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-26T14:16:57.437773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.437777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-26T14:16:57.437780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.437787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-26T14:16:57.438036Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-26T14:16:57.438049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.438052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.438057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.438129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00002? size 1048780 WTime 1764166617437
2025-11-26T14:16:57.438683Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.438697Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.442775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 8 size 7878063 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.442791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 536753 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.442807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.442932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.442939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.443081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-26T14:16:57.443089Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-26T14:16:57.444227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.444245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.444284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.444291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.444301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-26T14:16:57.444304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.444310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 1, Offset: 16 is stored on disk
2025-11-26T14:16:57.444316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.444321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 2, Offset: 16 is stored on disk
2025-11-26T14:16:57.444355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.444359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.444361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.444363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.444365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.444391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.444407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.444421Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.444429Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00002?(+) to d0000000000_00000000000000000008_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444431Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00002?(+) to d0000000000_00000000000000000009_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444434Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00002?(+) to d0000000000_00000000000000000010_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444436Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00002?(+) to d0000000000_00000000000000000011_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444438Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00002?(+) to d0000000000_00000000000000000012_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444441Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00002?(+) to d0000000000_00000000000000000013_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444443Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00002?(+) to d0000000000_00000000000000000014_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444445Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00002?(+) to d0000000000_00000000000000000015_00000_0000000001_00002?(+)
2025-11-26T14:16:57.444448Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00002_0000000001_00000|(+) to d0000000000_00000000000000000007_00002_0000000001_00000|(+)
2025-11-26T14:16:57.444752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.444955Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.445492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793412:3461] destroyed
2025-11-26T14:16:57.445948Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445956Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445981Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.445999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.446009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.446015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.446018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.446019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.446021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.446023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.446030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.446134Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446153Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446162Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.446182Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    PartNo: 0
    WriteTimestampMS: 1764166617437
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.460758Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.464332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793420:3468], now have 1 active actors on pipe
2025-11-26T14:16:57.464378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.464386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.464439Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|86af4ae8-1866c79b-9151df29-d289f30e_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.464452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.464456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.464462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.464465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.464469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.464481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.464483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.464487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.464503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.464519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.468429Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.469013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793420:3468] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|86af4ae8-1866c79b-9151df29-d289f30e_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|86af4ae8-1866c79b-9151df29-d289f30e_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.479767Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.481058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793424:3469], now have 1 active actors on pipe
2025-11-26T14:16:57.482407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.482427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.482503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.482562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.482573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.482594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.482622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.482640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.482644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.482653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.482655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.482660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.482662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.482666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.482696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-26T14:16:57.482708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.482712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-26T14:16:57.482715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.482718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-26T14:16:57.483041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 1048790 count 1 nextOffset 18 batches 3
2025-11-26T14:16:57.483052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.483055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.483058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.483126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00002? size 1048780 WTime 1764166617482
2025-11-26T14:16:57.483291Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.483308Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.494151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.494180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.494206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.494212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.494223Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-26T14:16:57.494226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.494231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 1, Offset: 17 is stored on disk
2025-11-26T14:16:57.494236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.494240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 2, Offset: 17 is stored on disk
2025-11-26T14:16:57.494277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.494280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.494282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.494285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.494287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.494295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.494311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.494446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.494545Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.495100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793424:3469] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    PartNo: 0
    WriteTimestampMS: 1764166617481
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.500366Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|16da6b8b-e22cbf07-b689acd5-bce9e148_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:57.501944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793429:3473], now have 1 active actors on pipe
2025-11-26T14:16:57.502138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.502149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.502189Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|16da6b8b-e22cbf07-b689acd5-bce9e148_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.502204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.502210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.502216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.502219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.502224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.502235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.502238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.502242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.502265Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.502287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.502379Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|16da6b8b-e22cbf07-b689acd5-bce9e148_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.507160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793429:3473] destroyed
2025-11-26T14:16:57.512340Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.528321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.528343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.528347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.528351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.528357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.528675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793432:3474], now have 1 active actors on pipe
2025-11-26T14:16:57.529965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.529975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.530047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.530092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.530102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.530106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.530120Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.530136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.530141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.530149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.530152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.530155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.530156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.530162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.530190Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-26T14:16:57.530203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.530206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-26T14:16:57.530210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.530213Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-26T14:16:57.530477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 1048790 count 1 nextOffset 19 batches 3
2025-11-26T14:16:57.530484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.530486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.530489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.530548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00002? size 1048780 WTime 1764166617528
2025-11-26T14:16:57.530731Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.530742Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 1048780
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    PartNo: 0
    WriteTimestampMS: 1764166617528
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 27
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.556764Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.556421Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.556466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.556493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.556503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.556514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-26T14:16:57.556519Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.556523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 1, Offset: 18 is stored on disk
2025-11-26T14:16:57.556528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.556532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 2, Offset: 18 is stored on disk
2025-11-26T14:16:57.556571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.556575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.556579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.556583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.556585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.556594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.556609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.556677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.557647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793432:3474] destroyed
2025-11-26T14:16:57.568346Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.576503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793437:3477], now have 1 active actors on pipe
2025-11-26T14:16:57.580514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.580530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.580562Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|21da188f-bddaabff-e17a3383-c1acf7bd_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.580572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.580575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.580579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.580580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.580585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.580593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.580595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.580598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.580612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.580628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.580791Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|21da188f-bddaabff-e17a3383-c1acf7bd_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|21da188f-bddaabff-e17a3383-c1acf7bd_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.584682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793437:3477] destroyed
2025-11-26T14:16:57.592246Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.608507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793441:3479], now have 1 active actors on pipe
2025-11-26T14:16:57.614000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.614028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.614102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.614154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.614167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.614172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.614191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.614213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.614218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.614227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.614230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.614235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.614237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.614242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.614272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-26T14:16:57.614285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.614288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-26T14:16:57.614294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.614297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-26T14:16:57.614621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 1048790 count 1 nextOffset 20 batches 3
2025-11-26T14:16:57.614632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.614635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.614639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.614714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00002? size 1048780 WTime 1764166617613
2025-11-26T14:16:57.615572Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.615586Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.619127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.619152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.619177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.619184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.619195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-26T14:16:57.619198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.619203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 1, Offset: 19 is stored on disk
2025-11-26T14:16:57.619208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.619213Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 2, Offset: 19 is stored on disk
2025-11-26T14:16:57.619248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.619253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.619255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.619258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.619260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.619268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.619285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.619418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.620037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793441:3479] destroyed
2025-11-26T14:16:57.619492Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    PartNo: 0
    WriteTimestampMS: 1764166617613
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.628420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.628435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.628438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.628441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.628443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.628343Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.635795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793447:3483], now have 1 active actors on pipe
2025-11-26T14:16:57.635846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.635857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.635902Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cb358e1-692d2952-5104e429-33ee73c2_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.635914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.635918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.635927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.635930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.635935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.635946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.635949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.635953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.635971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.635990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.640400Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cb358e1-692d2952-5104e429-33ee73c2_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:57.648338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793447:3483] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cb358e1-692d2952-5104e429-33ee73c2_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.656396Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.668347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793451:3485], now have 1 active actors on pipe
2025-11-26T14:16:57.673272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.673298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.673389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.673441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.673453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.673458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.673480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.673499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.673503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.673517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.673520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.673525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.673527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.673532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.673567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-26T14:16:57.673578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.673581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-26T14:16:57.673584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.673588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-26T14:16:57.673844Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 1048790 count 1 nextOffset 21 batches 3
2025-11-26T14:16:57.673852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.673855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.673859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.673923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00002? size 1048780 WTime 1764166617672
2025-11-26T14:16:57.674078Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.674090Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.678722Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    PartNo: 0
    WriteTimestampMS: 1764166617672
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.679968Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.678426Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.678452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.678476Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.678481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.678494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-26T14:16:57.678497Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.678502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 1, Offset: 20 is stored on disk
2025-11-26T14:16:57.678507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.678512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 2, Offset: 20 is stored on disk
2025-11-26T14:16:57.678552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.678557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.678559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.678561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.678563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.678572Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.678602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.678655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.679576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793451:3485] destroyed
2025-11-26T14:16:57.680507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793456:3488], now have 1 active actors on pipe
2025-11-26T14:16:57.680681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.680688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.680717Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b1918cd5-258588ba-3a5a6673-61a29a68_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.680726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.680731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.680735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.680737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.680740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.680749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.680751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.680754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.680767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.680780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.680901Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.681295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793456:3488] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b1918cd5-258588ba-3a5a6673-61a29a68_21"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b1918cd5-258588ba-3a5a6673-61a29a68_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.689083Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.689653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793459:3489], now have 1 active actors on pipe
2025-11-26T14:16:57.691039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.691058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.691128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.691179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.691192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.691196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.691212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.691231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.691235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.691244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.691246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.691250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.691253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.691256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.691285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-26T14:16:57.691296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.691299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-26T14:16:57.691302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.691304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-26T14:16:57.691622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 1048790 count 1 nextOffset 22 batches 3
2025-11-26T14:16:57.691632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.691634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.691638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.691704Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00002? size 1048780 WTime 1764166617690
2025-11-26T14:16:57.691888Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.691900Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.695661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.695683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.695756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.696232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.696272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.696281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.696292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-26T14:16:57.696295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.696300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 1, Offset: 21 is stored on disk
2025-11-26T14:16:57.696305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.696310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 2, Offset: 21 is stored on disk
2025-11-26T14:16:57.696357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.696359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.696360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.696363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.696365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.696375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.696399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.696612Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.697445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793459:3489] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    PartNo: 0
    WriteTimestampMS: 1764166617690
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.698246Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.698835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793464:3492], now have 1 active actors on pipe
2025-11-26T14:16:57.698957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.698966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.698995Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6ffd6859-4f427f3a-c0c39211-a514a1a7_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.699005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.699009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.699014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.699017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.699021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.699029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.699032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.699035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.699048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.699060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.699202Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.699623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793464:3492] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6ffd6859-4f427f3a-c0c39211-a514a1a7_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6ffd6859-4f427f3a-c0c39211-a514a1a7_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.710203Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.710963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793467:3493], now have 1 active actors on pipe
2025-11-26T14:16:57.714247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.714267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.714349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.714398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.714408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.714413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.714439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.714459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.714463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.714472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.714475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.714480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.714483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.714502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.714532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-26T14:16:57.714544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.714555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-26T14:16:57.714558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.714561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-26T14:16:57.714834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 1048790 count 1 nextOffset 23 batches 3
2025-11-26T14:16:57.714844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.714847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.714852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.714923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00002? size 1048780 WTime 1764166617710
2025-11-26T14:16:57.715179Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.715191Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.730526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.736442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.736486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.736516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.736524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.736535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-26T14:16:57.736539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.736545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 1, Offset: 22 is stored on disk
2025-11-26T14:16:57.736550Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.736556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 2, Offset: 22 is stored on disk
2025-11-26T14:16:57.736600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.736604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.736606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.736610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.736612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.736622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.736640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.736657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.736914Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    PartNo: 0
    WriteTimestampMS: 1764166617710
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 23
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.738104Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.740615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793467:3493] destroyed
2025-11-26T14:16:57.740633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793472:3496], now have 1 active actors on pipe
2025-11-26T14:16:57.750777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.750800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.750855Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c145f3b9-d39d8be7-c2f47933-e50ef79_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.750870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.750876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.750883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.750886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.750891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.750904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.750908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.750912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.750935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.750955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.751198Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c145f3b9-d39d8be7-c2f47933-e50ef79_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:57.751993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793472:3496] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c145f3b9-d39d8be7-c2f47933-e50ef79_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.764745Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.765470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793476:3498], now have 1 active actors on pipe
2025-11-26T14:16:57.766992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.767011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.767105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.767160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.767172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.767177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.767196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.767213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.767216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.767226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.767229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.767233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.767236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.767240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.767269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-26T14:16:57.767282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.767286Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-26T14:16:57.767290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.767293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-26T14:16:57.768041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 2 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-26T14:16:57.768054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.768056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.768059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.768135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002? size 1048780 WTime 1764166617768
2025-11-26T14:16:57.768472Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.768485Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.785112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.785143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.785167Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.785173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.785183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-26T14:16:57.785186Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.785190Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 1, Offset: 23 is stored on disk
2025-11-26T14:16:57.785195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.785198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 2, Offset: 23 is stored on disk
2025-11-26T14:16:57.785238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.785241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.785243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.785245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.785246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.785256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00002? (1048780)
2025-11-26T14:16:57.785276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-26T14:16:57.785292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.785299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-26T14:16:57.785301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-26T14:16:57.785308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-26T14:16:57.785314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-26T14:16:57.785321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-26T14:16:57.785327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-26T14:16:57.785335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-26T14:16:57.785340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-26T14:16:57.785345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-26T14:16:57.785350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-26T14:16:57.785358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785367Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.785383Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-26T14:16:57.785407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-26T14:16:57.785413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-26T14:16:57.785415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.785428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00002?, parameters.CurOffset=16
2025-11-26T14:16:57.785552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/3
2025-11-26T14:16:57.785558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-26T14:16:57.785562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-26T14:16:57.785700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.1/3
2025-11-26T14:16:57.785702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-26T14:16:57.785704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:1 appended
2025-11-26T14:16:57.785719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.2/3
2025-11-26T14:16:57.785720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-26T14:16:57.785933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-26T14:16:57.785937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:2 appended
2025-11-26T14:16:57.785940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-26T14:16:57.785941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.785942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00002?, parameters.CurOffset=17
2025-11-26T14:16:57.786098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/3
2025-11-26T14:16:57.786101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-26T14:16:57.786102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-26T14:16:57.786539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.1/3
2025-11-26T14:16:57.786541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-26T14:16:57.786543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:1 appended
2025-11-26T14:16:57.786559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.2/3
2025-11-26T14:16:57.786561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-26T14:16:57.786758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 2097545 count 2 nextOffset 18 batches 5
2025-11-26T14:16:57.786763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:2 appended
2025-11-26T14:16:57.786765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-26T14:16:57.786767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.786768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00002?, parameters.CurOffset=18
2025-11-26T14:16:57.786900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/3
2025-11-26T14:16:57.786902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-26T14:16:57.786904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-26T14:16:57.787192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.1/3
2025-11-26T14:16:57.787194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-26T14:16:57.787196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:1 appended
2025-11-26T14:16:57.787206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.2/3
2025-11-26T14:16:57.787208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-26T14:16:57.787372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 3146300 count 3 nextOffset 19 batches 7
2025-11-26T14:16:57.787376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:2 appended
2025-11-26T14:16:57.787378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-26T14:16:57.787379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.787380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00002?, parameters.CurOffset=19
2025-11-26T14:16:57.787501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/3
2025-11-26T14:16:57.787505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-26T14:16:57.787506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-26T14:16:57.787629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.1/3
2025-11-26T14:16:57.787631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-26T14:16:57.787634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:1 appended
2025-11-26T14:16:57.787644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.2/3
2025-11-26T14:16:57.787646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-26T14:16:57.787964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 4195055 count 4 nextOffset 20 batches 9
2025-11-26T14:16:57.787968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:2 appended
2025-11-26T14:16:57.787970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-26T14:16:57.787972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.787973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00002?, parameters.CurOffset=20
2025-11-26T14:16:57.788110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/3
2025-11-26T14:16:57.788112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-26T14:16:57.788113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-26T14:16:57.788244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.1/3
2025-11-26T14:16:57.788245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-26T14:16:57.788246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:1 appended
2025-11-26T14:16:57.788255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.2/3
2025-11-26T14:16:57.788340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.788355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788364Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:57.788446Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    PartNo: 0
    WriteTimestampMS: 1764166617762
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 22
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.789596Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.788257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-26T14:16:57.790056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 5243810 count 5 nextOffset 21 batches 11
2025-11-26T14:16:57.790063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:2 appended
2025-11-26T14:16:57.790070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-26T14:16:57.790072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.790075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00002?, parameters.CurOffset=21
2025-11-26T14:16:57.790227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/3
2025-11-26T14:16:57.790234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-26T14:16:57.790236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-26T14:16:57.790389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.1/3
2025-11-26T14:16:57.790403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-26T14:16:57.790405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:1 appended
2025-11-26T14:16:57.790418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.2/3
2025-11-26T14:16:57.790421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-26T14:16:57.790629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 6292565 count 6 nextOffset 22 batches 13
2025-11-26T14:16:57.790635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:2 appended
2025-11-26T14:16:57.790639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-26T14:16:57.790641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.790643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00002?, parameters.CurOffset=22
2025-11-26T14:16:57.790824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/3
2025-11-26T14:16:57.790829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-26T14:16:57.790831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-26T14:16:57.791171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.1/3
2025-11-26T14:16:57.791178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-26T14:16:57.791180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:1 appended
2025-11-26T14:16:57.791193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.2/3
2025-11-26T14:16:57.791195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-26T14:16:57.791411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 7341320 count 7 nextOffset 23 batches 15
2025-11-26T14:16:57.791416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:2 appended
2025-11-26T14:16:57.791420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-26T14:16:57.791421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.791424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00002?, parameters.CurOffset=23
2025-11-26T14:16:57.791570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/3
2025-11-26T14:16:57.791574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-26T14:16:57.793986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 23 partNo 0 result is x0000000000_00000000000000000015_00001_0000000008_00015 size 7878063
2025-11-26T14:16:57.793991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-26T14:16:57.795780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.1/3
2025-11-26T14:16:57.795792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-26T14:16:57.795797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:1 appended
2025-11-26T14:16:57.795817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.2/3
2025-11-26T14:16:57.795820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-26T14:16:57.795836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000015_00001_0000000008_00015 new key d0000000000_00000000000000000015_00001_0000000008_00015 size 7878063 WTime 1764166617795
2025-11-26T14:16:57.796082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 2 FormedBlobsCount 1 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-26T14:16:57.796089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:2 appended
2025-11-26T14:16:57.796142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 15 endOffset 16 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002| size 1048780 WTime 1764166617795
2025-11-26T14:16:57.796349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793476:3498] destroyed
2025-11-26T14:16:57.796360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793482:3502], now have 1 active actors on pipe
2025-11-26T14:16:57.796367Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.796380Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 8 size 7878063
2025-11-26T14:16:57.796388Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.796418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.796424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.796496Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9724311f-e099307e-9ab84e31-7caf0f99_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.796507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.796511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.796515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.796518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.796522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.796531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.796536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.796538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.796555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.796592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.799414Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9724311f-e099307e-9ab84e31-7caf0f99_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9724311f-e099307e-9ab84e31-7caf0f99_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.803401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793482:3502] destroyed
2025-11-26T14:16:57.811325Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.812017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793487:3505], now have 1 active actors on pipe
2025-11-26T14:16:57.813426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.813444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.813527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.813590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.813598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.813603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.813623Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.813641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.813644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.813653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.813655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.813659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.813661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.813665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.813694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-26T14:16:57.813704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.813707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-26T14:16:57.813710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.813713Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-26T14:16:57.813956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-26T14:16:57.813975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.813979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.813982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.814050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00002? size 1048780 WTime 1764166617813
2025-11-26T14:16:57.814206Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.814226Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.817504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 8 size 7878063 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.817517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.817532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.817749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.817753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.817887Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0' size 7878063
2025-11-26T14:16:57.817897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-26T14:16:57.817902Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-26T14:16:57.817957Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 0 size 8365339 cause it's been evicted from L2. Actual L1 size: 12
2025-11-26T14:16:57.817964Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-26T14:16:57.818615Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.818631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.818656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.818663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.818673Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-26T14:16:57.818676Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.818682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 1, Offset: 24 is stored on disk
2025-11-26T14:16:57.818686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.818692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 2, Offset: 24 is stored on disk
2025-11-26T14:16:57.818719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.818722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.818724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.818726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.818728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.818754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.818769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.818780Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.818789Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00002?(+) to d0000000000_00000000000000000016_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818791Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00002?(+) to d0000000000_00000000000000000017_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818793Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00002?(+) to d0000000000_00000000000000000018_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818795Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00002?(+) to d0000000000_00000000000000000019_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818797Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00002?(+) to d0000000000_00000000000000000020_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818799Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00002?(+) to d0000000000_00000000000000000021_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818801Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00002?(+) to d0000000000_00000000000000000022_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818803Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002?(+) to d0000000000_00000000000000000023_00000_0000000001_00002?(+)
2025-11-26T14:16:57.818805Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00001_0000000001_00001|(+) to d0000000000_00000000000000000015_00001_0000000001_00001|(+)
2025-11-26T14:16:57.819101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.819298Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    PartNo: 0
    WriteTimestampMS: 1764166617813
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.820787Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.821805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793487:3505] destroyed
2025-11-26T14:16:57.821870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793495:3510], now have 1 active actors on pipe
2025-11-26T14:16:57.821878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.821886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.821920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821940Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821943Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821952Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.821981Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|951147f8-ee1540b0-37e4fd0b-f6b8a23f_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.821990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.821993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.822007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.822017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.822020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.822025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.822027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.822030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.822038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.822041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.822043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.822055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.822062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.822071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.822261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.822310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-26T14:16:57.822479Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|951147f8-ee1540b0-37e4fd0b-f6b8a23f_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|951147f8-ee1540b0-37e4fd0b-f6b8a23f_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.828672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793495:3510] destroyed
2025-11-26T14:16:57.830012Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.831881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793499:3512], now have 1 active actors on pipe
2025-11-26T14:16:57.833354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.833375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.833455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.833502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.833512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.833516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.833535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.833554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.833558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.833567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.833572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.833576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.833578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.833582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.833608Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-26T14:16:57.833618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.833620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-26T14:16:57.833623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.833625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-26T14:16:57.833971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 1048790 count 1 nextOffset 26 batches 3
2025-11-26T14:16:57.833983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.833986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.833989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.834071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00002? size 1048780 WTime 1764166617829
2025-11-26T14:16:57.834239Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.834254Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.837861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.839014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.839035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.839058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.839065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.839076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-26T14:16:57.839080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.839101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 1, Offset: 25 is stored on disk
2025-11-26T14:16:57.839108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.839113Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 2, Offset: 25 is stored on disk
2025-11-26T14:16:57.839154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.839158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.839160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.839162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.839165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.839174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.839191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.839286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.839363Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.840087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793499:3512] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    PartNo: 0
    WriteTimestampMS: 1764166617829
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.842855Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.843549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793504:3516], now have 1 active actors on pipe
2025-11-26T14:16:57.843747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.843757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.843789Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3681fcad-31d961a8-3fd08b45-50f604a4_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.843799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.843816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.843822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.843824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.843829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.843838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.843841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.843845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.843862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.843878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.844079Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.844637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793504:3516] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3681fcad-31d961a8-3fd08b45-50f604a4_26"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3681fcad-31d961a8-3fd08b45-50f604a4_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.861535Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.862305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793507:3517], now have 1 active actors on pipe
2025-11-26T14:16:57.863773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.863795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.863879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.863925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.863935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.863940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.863962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.863984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.863991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.864001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.864019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.864024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.864027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.864034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.864063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-26T14:16:57.864076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.864080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-26T14:16:57.864084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.864090Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-26T14:16:57.864444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 1048790 count 1 nextOffset 27 batches 3
2025-11-26T14:16:57.864455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.864459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.864463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.864541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00002? size 1048780 WTime 1764166617863
2025-11-26T14:16:57.864706Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.864720Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.908970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.909003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.909032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.909044Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.909056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-26T14:16:57.909060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.909065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 1, Offset: 26 is stored on disk
2025-11-26T14:16:57.909071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.909077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 2, Offset: 26 is stored on disk
2025-11-26T14:16:57.909125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.909130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.909132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.909135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.909137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.909147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.909166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.909326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.909432Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    PartNo: 0
    WriteTimestampMS: 1764166617863
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 41
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.910507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793507:3517] destroyed
2025-11-26T14:16:57.916577Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.917223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793512:3520], now have 1 active actors on pipe
2025-11-26T14:16:57.917511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.917520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.917555Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f8d0cc36-8feb764b-f0b682f9-1aaf9585_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.917568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.917572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.917578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.917580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.917585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.917594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.917597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.917601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.917619Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.917639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f8d0cc36-8feb764b-f0b682f9-1aaf9585_27"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f8d0cc36-8feb764b-f0b682f9-1aaf9585_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.917779Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.922749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793512:3520] destroyed
2025-11-26T14:16:57.928088Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.928744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793518:3523], now have 1 active actors on pipe
2025-11-26T14:16:57.930252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.930276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.930367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.930413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.930424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.930429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.930448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.930473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.930478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.930488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.930492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.930497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.930499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.930506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.930538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-26T14:16:57.930551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.930554Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-26T14:16:57.930561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.930564Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-26T14:16:57.930837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 1048790 count 1 nextOffset 28 batches 3
2025-11-26T14:16:57.930849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.930852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.930855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.930935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00002? size 1048780 WTime 1764166617927
2025-11-26T14:16:57.931245Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.931260Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 1048780
2025-11-26T14:16:57.943554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.945678Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:57.945710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.945739Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.945747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.945761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-26T14:16:57.945765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.945770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 1, Offset: 27 is stored on disk
2025-11-26T14:16:57.945776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.945782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 2, Offset: 27 is stored on disk
2025-11-26T14:16:57.945857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.945861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.945863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.945865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.945867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.945878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.945897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.946037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:57.946534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793518:3523] destroyed
2025-11-26T14:16:57.946112Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    PartNo: 0
    WriteTimestampMS: 1764166617927
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 17
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.952397Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.953304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793533:3528], now have 1 active actors on pipe
2025-11-26T14:16:57.953485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.953499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.953536Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5b190328-423de820-6daf45f9-5a16c055_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.953548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.953552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.953557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.953559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.953563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.953574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.953577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.953581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.953597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.953615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.953754Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.954137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793533:3528] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5b190328-423de820-6daf45f9-5a16c055_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5b190328-423de820-6daf45f9-5a16c055_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:57.963838Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.972581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037924774793536:3532], now have 1 active actors on pipe
2025-11-26T14:16:57.977932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.977963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.978038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:57.978089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:57.978099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:57.978107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:57.978133Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.978156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.978160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.978169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.978171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.978175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:57.978176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.978181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.978210Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-26T14:16:57.978221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.978225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-26T14:16:57.978228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.978233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-26T14:16:57.978469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 1048790 count 1 nextOffset 29 batches 3
2025-11-26T14:16:57.978479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:57.978482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:57.978486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.978557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00002? size 1048780 WTime 1764166617976
2025-11-26T14:16:57.978738Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.978752Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 1048780
2025-11-26T14:16:58.004802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.004831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.004857Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.004865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.004877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-26T14:16:58.004881Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.004887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 1, Offset: 28 is stored on disk
2025-11-26T14:16:58.004893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.004899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 2, Offset: 28 is stored on disk
2025-11-26T14:16:58.004937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.004940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.004942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.004945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.004948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.004957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.004972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.005107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.005224Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    PartNo: 0
    WriteTimestampMS: 1764166617976
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 26
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.006144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037924774793536:3532] destroyed
2025-11-26T14:16:58.012651Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.013256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760838:3536], now have 1 active actors on pipe
2025-11-26T14:16:58.013467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.013478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.013512Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|45fe2f23-c06018c6-f53c510f-2b2644ef_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.013522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.013526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.013534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.013536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.013540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.013549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.013551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.013554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.013567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.013579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|45fe2f23-c06018c6-f53c510f-2b2644ef_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|45fe2f23-c06018c6-f53c510f-2b2644ef_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:58.013720Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.018126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760838:3536] destroyed
2025-11-26T14:16:58.023852Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.024463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760841:3537], now have 1 active actors on pipe
2025-11-26T14:16:58.025731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.025752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.025827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:58.025882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:58.025891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:58.025896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:58.025918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.025939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.025943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.025953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.025956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.025960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:58.025962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.025968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.025993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-26T14:16:58.026003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.026006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-26T14:16:58.026009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.026012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-26T14:16:58.026278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 1048790 count 1 nextOffset 30 batches 3
2025-11-26T14:16:58.026289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:58.026292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:58.026295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.026370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00002? size 1048780 WTime 1764166618024
2025-11-26T14:16:58.026575Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.026604Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 1048780
2025-11-26T14:16:58.036809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.036839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.036868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.036876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.036885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-26T14:16:58.036890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.036895Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 1, Offset: 29 is stored on disk
2025-11-26T14:16:58.036901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.036906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 2, Offset: 29 is stored on disk
2025-11-26T14:16:58.036944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.036948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.036950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.036952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.036954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.036964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.036980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.037126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.037237Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    PartNo: 0
    WriteTimestampMS: 1764166618024
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.038629Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.039302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760841:3537] destroyed
2025-11-26T14:16:58.039320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760846:3540], now have 1 active actors on pipe
2025-11-26T14:16:58.039457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.039465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.039497Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7c0bf3b6-d89d834-5badc3b4-af0da0c7_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.039508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.039512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.039518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.039521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.039524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.039533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.039537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.039545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.039558Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.039568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.039695Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7c0bf3b6-d89d834-5badc3b4-af0da0c7_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
2025-11-26T14:16:58.040396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760846:3540] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7c0bf3b6-d89d834-5badc3b4-af0da0c7_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:58.044359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.044374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.044377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.044380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.044382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.046837Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.047355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760849:3541], now have 1 active actors on pipe
2025-11-26T14:16:58.048671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.048690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.048763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:58.048814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:58.048824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:58.048828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:58.048849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.048868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.048875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.048884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.048886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.048890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:58.048892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.048897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.048924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-26T14:16:58.048935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.048938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-26T14:16:58.048942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.048946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-26T14:16:58.049181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 1048790 count 1 nextOffset 31 batches 3
2025-11-26T14:16:58.049191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:58.049194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:58.049201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.049273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00002? size 1048780 WTime 1764166618048
2025-11-26T14:16:58.049471Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.049484Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 1048780
2025-11-26T14:16:58.058018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.058048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.058073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.058080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.058093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-26T14:16:58.058096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.058101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 1, Offset: 30 is stored on disk
2025-11-26T14:16:58.058107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.058112Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 2, Offset: 30 is stored on disk
2025-11-26T14:16:58.058157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.058161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.058163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.058166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.058168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.058180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.058198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.058346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.058433Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.059003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760849:3541] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    PartNo: 0
    WriteTimestampMS: 1764166618048
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.064841Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.065473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760854:3544], now have 1 active actors on pipe
2025-11-26T14:16:58.065770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.065781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.065819Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a5cdbf3b-bb04017a-2d2efb32-60030862_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.065831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.065835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.065841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.065843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.065848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.065858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.065861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.065868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.065885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.065902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.066069Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.066593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760854:3544] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a5cdbf3b-bb04017a-2d2efb32-60030862_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a5cdbf3b-bb04017a-2d2efb32-60030862_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:58.077672Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.084504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760860:3545], now have 1 active actors on pipe
2025-11-26T14:16:58.085912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.085930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.085997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size: 511959
2025-11-26T14:16:58.086046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 1 messageNo: 0 size: 511959
2025-11-26T14:16:58.086053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1974: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size: 24658
2025-11-26T14:16:58.086058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-26T14:16:58.086082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.086101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.086105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.086115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.086117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.086121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-26T14:16:58.086123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.086127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.086156Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-26T14:16:58.086169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.086172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-26T14:16:58.086175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.086179Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-26T14:16:58.086393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 1048790 count 1 nextOffset 32 batches 3
2025-11-26T14:16:58.086401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-26T14:16:58.086405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-26T14:16:58.086410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.086472Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00002? size 1048780 WTime 1764166618084
2025-11-26T14:16:58.086670Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.086683Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 1048780
2025-11-26T14:16:58.108976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 1048780 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.109011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.109039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.109047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.109060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-26T14:16:58.109062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.109067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 1, Offset: 31 is stored on disk
2025-11-26T14:16:58.109073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.109076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 2, Offset: 31 is stored on disk
2025-11-26T14:16:58.109120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.109123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.109125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.109128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.109130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.109140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00002? (1048780)
2025-11-26T14:16:58.109161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-26T14:16:58.109177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.109191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-26T14:16:58.109194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-26T14:16:58.109208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-26T14:16:58.109217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-26T14:16:58.109227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-26T14:16:58.109235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-26T14:16:58.109252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-26T14:16:58.109263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-26T14:16:58.109280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-26T14:16:58.109290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-26T14:16:58.109302Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.109306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.109309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.109313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.109317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.109320Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.109323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.109326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.109334Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-26T14:16:58.109367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-26T14:16:58.109376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-26T14:16:58.109378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.109381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00002?, parameters.CurOffset=24
2025-11-26T14:16:58.109526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/3
2025-11-26T14:16:58.109535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-26T14:16:58.109542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
2025-11-26T14:16:58.109688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.1/3
2025-11-26T14:16:58.109694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-26T14:16:58.109697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:1 appended
2025-11-26T14:16:58.109714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.2/3
2025-11-26T14:16:58.109717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-26T14:16:58.109951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-26T14:16:58.109958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:2 appended
2025-11-26T14:16:58.109961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-26T14:16:58.109964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.109966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00002?, parameters.CurOffset=25
2025-11-26T14:16:58.110098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/3
2025-11-26T14:16:58.110105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-26T14:16:58.110108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-26T14:16:58.110241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.1/3
2025-11-26T14:16:58.110244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-26T14:16:58.110246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:1 appended
2025-11-26T14:16:58.110261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.2/3
2025-11-26T14:16:58.110264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-26T14:16:58.110448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 2097545 count 2 nextOffset 26 batches 5
2025-11-26T14:16:58.110453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:2 appended
2025-11-26T14:16:58.110456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-26T14:16:58.110458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.110462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00002?, parameters.CurOffset=26
2025-11-26T14:16:58.110626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/3
2025-11-26T14:16:58.110631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-26T14:16:58.110633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-26T14:16:58.110781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.1/3
2025-11-26T14:16:58.110784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-26T14:16:58.110786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:1 appended
2025-11-26T14:16:58.110799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.2/3
2025-11-26T14:16:58.110801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-26T14:16:58.110974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 3146300 count 3 nextOffset 27 batches 7
2025-11-26T14:16:58.110979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:2 appended
2025-11-26T14:16:58.110982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-26T14:16:58.110983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.110986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00002?, parameters.CurOffset=27
2025-11-26T14:16:58.111140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/3
2025-11-26T14:16:58.111143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-26T14:16:58.111146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-26T14:16:58.111299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.1/3
2025-11-26T14:16:58.111301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-26T14:16:58.111304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:1 appended
2025-11-26T14:16:58.111318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.2/3
2025-11-26T14:16:58.111323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-26T14:16:58.111525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 4195055 count 4 nextOffset 28 batches 9
2025-11-26T14:16:58.111530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:2 appended
2025-11-26T14:16:58.111533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-26T14:16:58.111535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.111538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00002?, parameters.CurOffset=28
2025-11-26T14:16:58.111680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/3
2025-11-26T14:16:58.111684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-26T14:16:58.111686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-26T14:16:58.111836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.1/3
2025-11-26T14:16:58.111867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-26T14:16:58.111869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:1 appended
2025-11-26T14:16:58.111880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.2/3
2025-11-26T14:16:58.111882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-26T14:16:58.112928Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.112754Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.112774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:58.112778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:58.112781Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:58.112785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:58.112787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:58.112791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:58.112794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63'
2025-11-26T14:16:58.112797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63'
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    PartNo: 0
    WriteTimestampMS: 1764166618084
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 24
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
>>>>> 1
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 0
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-26T14:16:58.114312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 5243810 count 5 nextOffset 29 batches 11
2025-11-26T14:16:58.114332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:2 appended
2025-11-26T14:16:58.114337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-26T14:16:58.114339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.114342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00002?, parameters.CurOffset=29
2025-11-26T14:16:58.114845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/3
2025-11-26T14:16:58.114852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-26T14:16:58.114856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-26T14:16:58.114994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.1/3
2025-11-26T14:16:58.114998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-26T14:16:58.114999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:1 appended
2025-11-26T14:16:58.115012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.2/3
2025-11-26T14:16:58.115014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-26T14:16:58.115203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 6292565 count 6 nextOffset 30 batches 13
2025-11-26T14:16:58.115208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:2 appended
2025-11-26T14:16:58.115212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-26T14:16:58.115214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.115217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00002?, parameters.CurOffset=30
2025-11-26T14:16:58.115367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/3
2025-11-26T14:16:58.115370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-26T14:16:58.115372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-26T14:16:58.115519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.1/3
2025-11-26T14:16:58.115522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-26T14:16:58.115524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:1 appended
2025-11-26T14:16:58.115535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.2/3
2025-11-26T14:16:58.115538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-26T14:16:58.124366Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.133103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 30 partNo 2 result is x0000000000_00000000000000000023_00000_0000000007_00016 size 8365364
2025-11-26T14:16:58.133135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000023_00000_0000000007_00016 new key d0000000000_00000000000000000023_00000_0000000007_00016 size 8365364 WTime 1764166618132
2025-11-26T14:16:58.133169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 2 FormedBlobsCount 1 NewHead: Offset 30 PartNo 2 PackedSize 24736 count 1 nextOffset 31 batches 1
2025-11-26T14:16:58.133176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:2 appended
2025-11-26T14:16:58.133182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-26T14:16:58.133185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.133189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00002?, parameters.CurOffset=31
2025-11-26T14:16:58.133318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/3
2025-11-26T14:16:58.133324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-26T14:16:58.133327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-26T14:16:58.133458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.1/3
2025-11-26T14:16:58.133461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-26T14:16:58.133462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:1 appended
2025-11-26T14:16:58.133478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.2/3
2025-11-26T14:16:58.133480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-26T14:16:58.133661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 2 PackedSize 1073491 count 2 nextOffset 32 batches 3
2025-11-26T14:16:58.133666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:2 appended
2025-11-26T14:16:58.133713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,2 HeadOffset 23 endOffset 24 curOffset 32 d0000000000_00000000000000000030_00002_0000000002_00002| size 1073481 WTime 1764166618132
2025-11-26T14:16:58.133900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760860:3545] destroyed
2025-11-26T14:16:58.133908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760867:3550], now have 1 active actors on pipe
2025-11-26T14:16:58.133935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.133942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.133955Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.133966Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 7 size 8365364
2025-11-26T14:16:58.133973Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 2 count 2 size 1073481
2025-11-26T14:16:58.134054Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 0
2025-11-26T14:16:58.134253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 3 blobs, size 25145475 count 24 last offset 15, current partition end offset: 32
2025-11-26T14:16:58.134257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-26T14:16:58.134269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 7 parts_count 16 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.134274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 2 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.134278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.134310Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 4. Have to read 1 of 3 from KV
2025-11-26T14:16:58.134641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0'
2025-11-26T14:16:58.134650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-26T14:16:58.134655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 7 parts_count 16
2025-11-26T14:16:58.145977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.145995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.145997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.146000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.146002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.208226Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 3 from KV. Status 1
2025-11-26T14:16:58.216331Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-11-26T14:16:58.216365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.216469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 7 size 8365364 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.216475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 2 size 1073481 actorID [2:7577037920725865808:2357]
2025-11-26T14:16:58.216499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 3 blobs
2025-11-26T14:16:58.216687Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.216832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.216877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-26T14:16:58.216888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-26T14:16:58.216909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216914Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216918Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216932Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0' size 8365364
2025-11-26T14:16:58.216950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-26T14:16:58.216954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-26T14:16:58.216957Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 2 count 2 parts 2 suffix '124' size 1073481
2025-11-26T14:16:58.217058Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.217185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.217358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.217495Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.217821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.217954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.218092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.218220Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.218553Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.218882Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.219040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.219184Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.219204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:58.219824Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-26T14:16:58.219851Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:58.219980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.220102Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.220239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.224471Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.224675Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.224823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.224968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.225104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.225257Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.225413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.225566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.225726Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.225877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.226010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.226028Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:58.226161Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.226335Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.226349Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:58.226426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.226643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.226647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.226649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.226652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.226654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.226682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.226719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 23 size 1048780 cause it's been evicted from L2. Actual L1 size: 13
2025-11-26T14:16:58.226725Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-26T14:16:58.226740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 29 size 1048780 cause it's been evicted from L2. Actual L1 size: 12
2025-11-26T14:16:58.226743Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 26 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-26T14:16:58.226746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 30 size 1048780 cause it's been evicted from L2. Actual L1 size: 10
2025-11-26T14:16:58.226749Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 25 size 1048780 cause it's been evicted from L2. Actual L1 size: 9
2025-11-26T14:16:58.226752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 27 size 1048780 cause it's been evicted from L2. Actual L1 size: 8
2025-11-26T14:16:58.226755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 31 size 1048780 cause it's been evicted from L2. Actual L1 size: 7
2025-11-26T14:16:58.226758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 7 size 7878063 cause it's been evicted from L2. Actual L1 size: 6
2025-11-26T14:16:58.226761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 24 size 1048780 cause it's been evicted from L2. Actual L1 size: 5
2025-11-26T14:16:58.226763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 28 size 1048780 cause it's been evicted from L2. Actual L1 size: 4
2025-11-26T14:16:58.226765Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-26T14:16:58.226824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.226857Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.226864Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00002?(+) to d0000000000_00000000000000000024_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226867Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00002?(+) to d0000000000_00000000000000000025_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226869Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00002?(+) to d0000000000_00000000000000000026_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226872Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00002?(+) to d0000000000_00000000000000000027_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226874Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00002?(+) to d0000000000_00000000000000000028_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226877Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00002?(+) to d0000000000_00000000000000000029_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226879Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00002?(+) to d0000000000_00000000000000000030_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226882Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00002?(+) to d0000000000_00000000000000000031_00000_0000000001_00002?(+)
2025-11-26T14:16:58.226884Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002|(+) to d0000000000_00000000000000000023_00000_0000000001_00002|(+)
2025-11-26T14:16:58.255350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:58.268983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.269014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.269023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.269027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.269030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.269032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.269034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.269041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.274799Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.308556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760867:3550] destroyed
CallPersQueueGRPC response:
2025-11-26T14:16:58.356207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.356233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.356236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.356239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.356241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 0
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
>>>>> 2
CallPersQueueGRPC request to localhost:16536
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 16
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-26T14:16:58.448226Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.450049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037929069760926:3580], now have 1 active actors on pipe
2025-11-26T14:16:58.450092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.450101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.450144Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user offset 16 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 16
2025-11-26T14:16:58.450439Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 9438782 count 16 last offset 23, current partition end offset: 32
2025-11-26T14:16:58.450445Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-26T14:16:58.450463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 1 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.450468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 7 parts_count 16 source 1 size 8365364 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.450484Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-26T14:16:58.450526Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-26T14:16:58.450963Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.451187Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.451359Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.451550Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.451695Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.451958Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.451992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-26T14:16:58.451998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0'
2025-11-26T14:16:58.452108Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.452421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.452631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.452966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.453117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.453410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.453599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.453752Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.453771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:58.453957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.454103Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.454120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:58.454264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.454410Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.454567Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.454735Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.454913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.455063Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.455246Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.455405Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.455582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.455732Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.455913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-26T14:16:58.456068Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-26T14:16:58.456087Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-26T14:16:58.456422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-26T14:16:58.456535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.468421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.468443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.468446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.468450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.468455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.476729Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.491282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037929069760926:3580] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 16
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-26T14:16:58.569068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.569089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.569091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.569095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.569098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.672545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.672570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.672573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.672577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.672580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.776410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.776431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.776434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.776437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.776445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.877354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.877375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.877378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.877382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.877385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.984358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.984384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.984386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.984391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.984393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:59.084412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:59.084436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.084439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:59.084443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.084447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:59.188411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:59.188433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.188435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:59.188438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.188441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:59.289944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:59.289968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.289970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:59.289975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.289978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:59.389589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:59.389612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.389616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:59.389620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.389622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3269, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext &): (info0.BlobsFromCache == 1) failed: (2 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x150A84EC)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x152644F0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext&)+4375 (0x14DD3BE7)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14EBF347)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x152666C8)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14EBEC44)
NUnitTest::TTestFactory::Execute()+817 (0x15266E11)
NUnitTest::RunMain(int, char**)+3229 (0x1527927D)
??+0 (0x7F2AC9AB2D90)
__libc_start_main+128 (0x7F2AC9AB2E40)
_start+41 (0x13EC7029)
[[rst]]
2025-11-26T14:16:49.914066Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7577037892629028168:2266];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:49.914098Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-26T14:16:49.923180Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7577037892561629737:2087];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:49.923291Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/isxu/000933/r3tmp/tmp4Vl7yJ/pdisk_1.dat
2025-11-26T14:16:49.925063Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-26T14:16:49.925060Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-26T14:16:49.978890Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-26T14:16:49.994982Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-26T14:16:50.016164Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-26T14:16:50.016188Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-26T14:16:50.017560Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-26T14:16:50.020220Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-11-26T14:16:50.029140Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-26T14:16:50.029168Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-26T14:16:50.030071Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-26T14:16:50.030346Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 28333, node 1
2025-11-26T14:16:50.105701Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/isxu/000933/r3tmp/yandext8jVCl.tmp
2025-11-26T14:16:50.105718Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/isxu/000933/r3tmp/yandext8jVCl.tmp
2025-11-26T14:16:50.105788Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/isxu/000933/r3tmp/yandext8jVCl.tmp
2025-11-26T14:16:50.105842Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-26T14:16:50.159207Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-26T14:16:50.288038Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-26T14:16:50.296913Z INFO: TTestServer started on Port 20251 GrpcPort 28333
TClient is connected to server localhost:20251
PQClient connected to localhost:28333
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-26T14:16:50.397549Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
2025-11-26T14:16:50.403358Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577037896856597322:2296], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.403403Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
waiting...
2025-11-26T14:16:50.408485Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577037896856597334:2299], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.408520Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577037896856597335:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.408583Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-26T14:16:50.414236Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
waiting...
2025-11-26T14:16:50.434091Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7577037896856597338:2301], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715657 completed, doublechecking }
2025-11-26T14:16:50.440781Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
2025-11-26T14:16:50.525176Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7577037896856597427:2170] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 5], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
waiting...
waiting...
2025-11-26T14:16:50.693190Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7577037896856597434:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-26T14:16:50.693718Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=OWUwZDBmZTItOTRjNWEzNDMtMjRmNGU0ZTktOWE3NjI3NWI=, ActorId: [2:7577037896856597320:2295], ActorState: ExecuteState, TraceId: 01kb08cbf0cz9cfmfzb17k1gge, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-26T14:16:50.694299Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-26T14:16:50.693136Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7577037896923996182:2324], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-26T14:16:50.697122Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=YWJkYTQzNmEtZTEyN2ZmOGQtOGEzNmIyNTMtNGVjYzUwYmY=, ActorId: [1:7577037896923996132:2316], ActorState: ExecuteState, TraceId: 01kb08cbgfe3aemy3a0qhbm945, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-26T14:16:50.697253Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-26T14:16:50.736495Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-26T14:16:50.780255Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-26T14:16:50.875790Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
2025-11-26T14:16:50.914621Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-26T14:16:50.924876Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-26T14:16:51.050971Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720665. Ctx: { TraceId: 01kb08cbz7chmqnf2txkpkpcws, Database: , SessionId: ydb://session/3?node_id=1&id=YWE2YjViYjktMTExZTFhYjktMzJjNTM1MGItZjhiYjk5NGI=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7577037901218964064:3034] 
2025-11-26T14:16:54.916871Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7577037892629028168:2266];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:54.916915Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-26T14:16:54.924944Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7577037892561629737:2087];send_to=[0:7307199536658146131:7762515];
2025-11-26T14:16:54.924974Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:28333
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-26T14:16:56.276820Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:28333
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-26T14:16:56.324301Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7577037922693800965:3262] connected; active server actors: 1
2025-11-26T14:16:56.324392Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-26T14:16:56.328323Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:971: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-26T14:16:56.332920Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-26T14:16:56.333393Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720672. Ctx: { TraceId: 01kb08ch7kc6tmbdtv60b9bgq1, Database: , SessionId: ydb://session/3?node_id=1&id=NWQxYTJiNTUtNmViYjFkMjAtY2FhNjRmN2YtNGQzNDMwYWI=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-11-26T14:16:56.336598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2957: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-26T14:16:56.344744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2989: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-26T14:16:56.344836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:785: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-26T14:16:56.344854Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:911: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-26T14:16:56.344867Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-26T14:16:56.348418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693800964:3261], now have 1 active actors on pipe
2025-11-26T14:16:56.348430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693800990:2452], now have 1 active actors on pipe
2025-11-26T14:16:56.348736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.352339Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-26T14:16:56.356875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
2025-11-26T14:16:56.369749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801031:3308], now have 1 active actors on pipe
2025-11-26T14:16:56.370004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--topic1 version0
2025-11-26T14:16:56.376785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.377026Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-26T14:16:56.377046Z node 2 :PERSQUEUE DEBUG: partition.cpp:1315: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-26T14:16:56.377129Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-26T14:16:56.377211Z node 2 :PERSQUEUE INFO: partition_init.cpp:1107: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7577037922626401495:2365]
2025-11-26T14:16:56.377977Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-26T14:16:56.377981Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-26T14:16:56.377987Z node 2 :PERSQUEUE INFO: partition.cpp:707: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7577037922626401495:2365]
2025-11-26T14:16:56.377995Z node 2 :PERSQUEUE DEBUG: partition.cpp:727: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-26T14:16:56.378004Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-26T14:16:56.378348Z node 2 :PERSQUEUE DEBUG: partition.cpp:4404: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-26T14:16:56.378355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.378368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2392: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-26T14:16:56.378389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.378391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.378394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.378396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.378414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.378416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.378419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.378422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.378423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.378432Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-26T14:16:56.378440Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-26T14:16:56.378525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.378561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.380576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.380628Z node 2 :PERSQUEUE DEBUG: partition.cpp:1420: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764166616415, TxId 281474976720671
2025-11-26T14:16:56.380632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.380634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.380636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.380641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2456: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-26T14:16:56.380661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.380663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.380666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.380711Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.397521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.397594Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-26T14:16:56.397651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.397665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.397669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.397672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.397675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.397677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.397687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.397823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 6291456 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-26T14:16:56.397853Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:911: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-26T14:16:56.397928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.404386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.404432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3567: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-26T14:16:56.411378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1043: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-26T14:16:56.481070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.481090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.481092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.481099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.481102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.584326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.584347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.584351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.584356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.584359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.688330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.688350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.688352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.688355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.688357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.792332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.792349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.792353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.792356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.792363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764166616415
    ParentPathId: 6
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764166616415
    ParentPathId: 6
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:28333
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-26T14:16:56.892071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.892089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.892092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.892095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.892098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.892619Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 6291456
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:56.894966Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.896499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801086:3353], now have 1 active actors on pipe
2025-11-26T14:16:56.900448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.900482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.900525Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4a5e40a6-b0b988ab-aacb9ae2-ca6d480b_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.900538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.900543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.900549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.900551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.900559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.900569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.900573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.900579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.900595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.900613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4a5e40a6-b0b988ab-aacb9ae2-ca6d480b_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4a5e40a6-b0b988ab-aacb9ae2-ca6d480b_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:56.900871Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.904436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037922693801086:3353] destroyed
2025-11-26T14:16:56.908275Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.912486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801089:3354], now have 1 active actors on pipe
2025-11-26T14:16:56.920486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.920511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.920640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:56.920658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:56.920675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.920680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.920692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.920695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.920700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.920728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-26T14:16:56.920788Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-26T14:16:56.920795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.920798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.920804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.921103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 262204 WTime 1764166616920
2025-11-26T14:16:56.921141Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.921162Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    WriteTimestampMS: 1764166616920
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 22
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:56.942845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.942231Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:56.942259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.942285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.942296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.942308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-26T14:16:56.942347Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-26T14:16:56.942353Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-26T14:16:56.942358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.942361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.942364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.942366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.942368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.942376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.942388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.942430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-26T14:16:56.942447Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 262204 count 1 last offset 0, current partition end offset: 1
2025-11-26T14:16:56.942450Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-26T14:16:56.942461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:56.942466Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-26T14:16:56.942481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-26T14:16:56.942639Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 262182 from pos 0 cbcount 1
2025-11-26T14:16:56.942659Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764166616920 queuesize 0 startOffset 0
2025-11-26T14:16:56.942720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:56.942730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:56.943717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037922693801089:3354] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1efd6c47-b6d2d8ef-ba14f645-cb7a5a0e_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1efd6c47-b6d2d8ef-ba14f645-cb7a5a0e_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:56.944703Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.945445Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.947990Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.945141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801100:3358], now have 1 active actors on pipe
2025-11-26T14:16:56.945276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.945282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.945311Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1efd6c47-b6d2d8ef-ba14f645-cb7a5a0e_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.945320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.945323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.945328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.945329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.945333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.945342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.945344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.945346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.945359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.945372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.947941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037922693801100:3358] destroyed
2025-11-26T14:16:56.948418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801103:3359], now have 1 active actors on pipe
2025-11-26T14:16:56.948956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.948973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.948995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:56.949004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:56.949017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.949020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.949029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.949031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.949036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.949062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-26T14:16:56.949098Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 262214 count 1 nextOffset 2 batches 1
2025-11-26T14:16:56.949104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.949106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.949109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.949560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00000? size 262204 WTime 1764166616949
2025-11-26T14:16:56.949598Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.949609Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    WriteTimestampMS: 1764166616948
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4af1bdd1-43a159c7-8915dd53-81a0e5f4_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4af1bdd1-43a159c7-8915dd53-81a0e5f4_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:56.960719Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.962012Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.962777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.960411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:56.960445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.960469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.960481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.960491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-26T14:16:56.960534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.960537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.960539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.960541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.960543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.960554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.960567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.960639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:56.961965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037922693801103:3359] destroyed
2025-11-26T14:16:56.962460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801110:3364], now have 1 active actors on pipe
2025-11-26T14:16:56.962605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.962610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.962639Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4af1bdd1-43a159c7-8915dd53-81a0e5f4_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.962649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.962652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.962656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.962657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.962660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.962671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.962673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.962676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.962687Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.962698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.966250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037922693801110:3364] destroyed
2025-11-26T14:16:56.966337Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.968705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801113:3365], now have 1 active actors on pipe
2025-11-26T14:16:56.972677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.972689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.972712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:56.972722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:56.972739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.972742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.972754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.972756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.972761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.972791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-26T14:16:56.972829Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 262214 count 1 nextOffset 3 batches 1
2025-11-26T14:16:56.972836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.972838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.972840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.973282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00000? size 262204 WTime 1764166616972
2025-11-26T14:16:56.973317Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:56.973328Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 262204
2025-11-26T14:16:56.981002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:56.981028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:56.981054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:56.981066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:56.981076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-26T14:16:56.981122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.981126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.981128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.981131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.981133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.981142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:56.981155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.981295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:56.981848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037922693801113:3365] destroyed
2025-11-26T14:16:56.981404Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    WriteTimestampMS: 1764166616972
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:56.989159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037922693801118:3368], now have 1 active actors on pipe
2025-11-26T14:16:56.989349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:56.989358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:56.989392Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3d50e8fa-a01c4d50-66899aa7-24512627_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:56.989404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.989408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:56.989414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:56.989416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.989420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:56.989431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:56.989435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:56.989438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:56.989457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:56.989474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:56.989906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037922693801118:3368] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3d50e8fa-a01c4d50-66899aa7-24512627_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3d50e8fa-a01c4d50-66899aa7-24512627_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:56.988632Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.989592Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:56.994004Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:56.994120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:56.994130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.994132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:56.994135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:56.994136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.001997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768417:3369], now have 1 active actors on pipe
2025-11-26T14:16:57.003000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.003010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.003032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.003042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.003060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.003065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.003078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.003080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.003084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.003114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-26T14:16:57.003150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 262214 count 1 nextOffset 4 batches 1
2025-11-26T14:16:57.003157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.003160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.003163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.003280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00000? size 262204 WTime 1764166617003
2025-11-26T14:16:57.003312Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.003323Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 262204
2025-11-26T14:16:57.017483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.017513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.017538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.017550Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.017561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-26T14:16:57.017581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.017607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.017610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.017612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.017616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.017618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.017628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.017638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.018126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768417:3369] destroyed
2025-11-26T14:16:57.017718Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    WriteTimestampMS: 1764166617002
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 14
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.024588Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.025058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768422:3372], now have 1 active actors on pipe
2025-11-26T14:16:57.025227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.025235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.025268Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|17f04038-ee687d9a-a792edf7-50b0c8f6_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.025282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.025286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.025291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.025293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.025296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.025307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.025310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.025313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.025331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.025346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.025477Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.025833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768422:3372] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|17f04038-ee687d9a-a792edf7-50b0c8f6_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|17f04038-ee687d9a-a792edf7-50b0c8f6_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.028170Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.028924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768425:3373], now have 1 active actors on pipe
2025-11-26T14:16:57.029463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.029474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.029493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.029502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.029517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.029520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.029529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.029531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.029535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.029559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-26T14:16:57.029586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 262214 count 1 nextOffset 5 batches 1
2025-11-26T14:16:57.029591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.029594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.029597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.029705Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00000? size 262204 WTime 1764166617029
2025-11-26T14:16:57.029735Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.029746Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 262204
2025-11-26T14:16:57.032290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.032313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.032334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.032343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.032352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-26T14:16:57.032381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.032384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.032385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.032387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.032389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.032397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.032407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.032541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.032605Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.033054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768425:3373] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    WriteTimestampMS: 1764166617029
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.033800Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.034406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768429:3375], now have 1 active actors on pipe
2025-11-26T14:16:57.034418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.034423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.034454Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cbab4755-8d7fcf0f-62566664-8cf59aca_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.034463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.034467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.034471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.034472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.034476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.034484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.034486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.034489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.034500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.034510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.034638Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.034981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768429:3375] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cbab4755-8d7fcf0f-62566664-8cf59aca_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cbab4755-8d7fcf0f-62566664-8cf59aca_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.037271Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.037745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768432:3376], now have 1 active actors on pipe
2025-11-26T14:16:57.038174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.038183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.038201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.038209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.038221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.038225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.038234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.038236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.038238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.038264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-26T14:16:57.038308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 262214 count 1 nextOffset 6 batches 1
2025-11-26T14:16:57.038313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.038316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.038318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.038426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00000? size 262204 WTime 1764166617036
2025-11-26T14:16:57.038457Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.038469Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 262204
2025-11-26T14:16:57.042063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.042081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.042098Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.042115Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.042122Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-26T14:16:57.042154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.042156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.042159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.042161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.042164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.042170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.042177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.042349Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.042414Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.042902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768432:3376] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    WriteTimestampMS: 1764166617036
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.043539Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.044046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768436:3378], now have 1 active actors on pipe
2025-11-26T14:16:57.044207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.044214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.044241Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|409e3c46-7a0614e3-a4cc6cc2-d22724e8_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.044248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.044250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.044254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.044256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.044272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.044280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.044282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.044285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.044296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.044306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.044420Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.044746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768436:3378] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|409e3c46-7a0614e3-a4cc6cc2-d22724e8_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|409e3c46-7a0614e3-a4cc6cc2-d22724e8_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.046995Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.047584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768439:3379], now have 1 active actors on pipe
2025-11-26T14:16:57.047985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.047993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.048012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.048019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.048030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.048033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.048041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.048043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.048046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.048072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-26T14:16:57.048111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 262214 count 1 nextOffset 7 batches 1
2025-11-26T14:16:57.048116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.048119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.048123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.048238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00000? size 262204 WTime 1764166617048
2025-11-26T14:16:57.048294Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.048306Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 262204
2025-11-26T14:16:57.051521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.051536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.051555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.051563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.051571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-26T14:16:57.051597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.051599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.051601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.051603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.051605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.051615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.051623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.051719Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.051807Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    WriteTimestampMS: 1764166617047
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.052318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768439:3379] destroyed
2025-11-26T14:16:57.052781Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.053458Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f4b06fe2-384ed84a-2b1c8897-2d286fc9_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f4b06fe2-384ed84a-2b1c8897-2d286fc9_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.056241Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.053183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768446:3381], now have 1 active actors on pipe
2025-11-26T14:16:57.053322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.053326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.053348Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f4b06fe2-384ed84a-2b1c8897-2d286fc9_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.053353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.053356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.053359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.053360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.053362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.053367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.053369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.053371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.053379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.053387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.055116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768446:3381] destroyed
2025-11-26T14:16:57.056619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768449:3382], now have 1 active actors on pipe
2025-11-26T14:16:57.057032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.057037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.057054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.057061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.057073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.057075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.057083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.057085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.057088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.057106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-26T14:16:57.057127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 262214 count 1 nextOffset 8 batches 1
2025-11-26T14:16:57.057132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.057135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.057137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.057241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00000? size 262204 WTime 1764166617056
2025-11-26T14:16:57.057291Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.057302Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 262204
2025-11-26T14:16:57.059849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.059867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.059883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.059891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.059899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-26T14:16:57.059929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.059931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.059933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.059935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.059937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.059942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.059953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.060053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.060168Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    WriteTimestampMS: 1764166617056
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.061052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768449:3382] destroyed
2025-11-26T14:16:57.061129Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.061536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768453:3384], now have 1 active actors on pipe
2025-11-26T14:16:57.061678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.061684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.061708Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|db5f91b1-ac325299-c7e2ba14-1285a942_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.061713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.061715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.061719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.061721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.061723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.061729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.061731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.061734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.061743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.061750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.061862Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|db5f91b1-ac325299-c7e2ba14-1285a942_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|db5f91b1-ac325299-c7e2ba14-1285a942_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.063502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768453:3384] destroyed
2025-11-26T14:16:57.064090Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.064517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768456:3385], now have 1 active actors on pipe
2025-11-26T14:16:57.064894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.064902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.064917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.064924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.064933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.064936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.064943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.064945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.064947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.064965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-26T14:16:57.064985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 262214 count 1 nextOffset 9 batches 1
2025-11-26T14:16:57.064990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.064992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.064994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.065092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00000? size 262204 WTime 1764166617064
2025-11-26T14:16:57.065121Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.065131Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 262204
2025-11-26T14:16:57.067496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.067510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.067526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.067532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.067539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-26T14:16:57.067564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.067566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.067568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.067569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.067571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.067577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.067584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.067680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.067762Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    WriteTimestampMS: 1764166617064
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.068620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768456:3385] destroyed
2025-11-26T14:16:57.068813Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.069262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768460:3387], now have 1 active actors on pipe
2025-11-26T14:16:57.069439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.069446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.069472Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f4230b89-d1880528-6a72ba16-76e51216_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.069478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.069481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.069484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.069485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.069488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.069494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.069496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.069498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.069507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.069516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.069638Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f4230b89-d1880528-6a72ba16-76e51216_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f4230b89-d1880528-6a72ba16-76e51216_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.071961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768460:3387] destroyed
2025-11-26T14:16:57.072039Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.072442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768463:3388], now have 1 active actors on pipe
2025-11-26T14:16:57.072840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.072847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.072863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.072870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.072880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.072883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.072890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.072891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.072894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.072915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-26T14:16:57.072940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 262214 count 1 nextOffset 10 batches 1
2025-11-26T14:16:57.072945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.072950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.072951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.073054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00000? size 262204 WTime 1764166617072
2025-11-26T14:16:57.073084Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.073095Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 262204
2025-11-26T14:16:57.076335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.076349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.076363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.076367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.076373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-26T14:16:57.076393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.076395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.076397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.076399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.076400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.076404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.076410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.076494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.076584Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.076917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768463:3388] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    WriteTimestampMS: 1764166617072
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.077403Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.077835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768467:3390], now have 1 active actors on pipe
2025-11-26T14:16:57.077997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.078003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.078027Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|31a4442d-5c992884-ce433893-6e362d55_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.078033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.078035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.078038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.078039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.078042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.078047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.078049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.078052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.078060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.078068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.078219Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.078534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768467:3390] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|31a4442d-5c992884-ce433893-6e362d55_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|31a4442d-5c992884-ce433893-6e362d55_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.080451Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.080867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768470:3391], now have 1 active actors on pipe
2025-11-26T14:16:57.081233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.081240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.081256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.081264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.081273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.081275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.081282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.081283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.081285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.081304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-26T14:16:57.081327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 262214 count 1 nextOffset 11 batches 1
2025-11-26T14:16:57.081332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.081334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.081337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.081436Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00000? size 262204 WTime 1764166617080
2025-11-26T14:16:57.081464Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.081474Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 262204
2025-11-26T14:16:57.084224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.084237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.084301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.084563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.084581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.084587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.084594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-26T14:16:57.084617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.084619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.084621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.084623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.084625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.084630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.084639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.084796Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.085204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768470:3391] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    WriteTimestampMS: 1764166617080
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.085699Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.086323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768474:3393], now have 1 active actors on pipe
2025-11-26T14:16:57.086334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.086338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.086362Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3b763dbb-8f2da940-7f3a76f4-90db8d9f_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.086367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.086370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.086373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.086374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.086377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.086383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.086385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.086387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.086395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.086404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.086516Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.086882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768474:3393] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3b763dbb-8f2da940-7f3a76f4-90db8d9f_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3b763dbb-8f2da940-7f3a76f4-90db8d9f_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.089165Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.089572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768477:3394], now have 1 active actors on pipe
2025-11-26T14:16:57.090004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.090012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.090030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.090039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.090048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.090050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.090057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.090059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.090062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.090084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-26T14:16:57.090118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 262214 count 1 nextOffset 12 batches 1
2025-11-26T14:16:57.090134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.090136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.090138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.090747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00000? size 262204 WTime 1764166617089
2025-11-26T14:16:57.090780Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.090790Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 262204
2025-11-26T14:16:57.094244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.094260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.094287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.094295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.094306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-26T14:16:57.094330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.094333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.094334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.094336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.094337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.094347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.094355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.094460Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.094552Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    WriteTimestampMS: 1764166617089
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.095773Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.096347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.096353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.096354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.096356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.096357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.096367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768477:3394] destroyed
2025-11-26T14:16:57.096372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768481:3396], now have 1 active actors on pipe
2025-11-26T14:16:57.096379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.096383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.096405Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c0d095f3-40c5f1de-24420e95-71eb83ad_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.096410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.096412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.096415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.096417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.096419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.096424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.096426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.096428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.096436Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.096444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.096589Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c0d095f3-40c5f1de-24420e95-71eb83ad_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c0d095f3-40c5f1de-24420e95-71eb83ad_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.097957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768481:3396] destroyed
2025-11-26T14:16:57.100553Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.101862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768484:3397], now have 1 active actors on pipe
2025-11-26T14:16:57.101951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.101956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.101973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.101981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.101991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.101996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.102004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.102006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.102010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.102030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-26T14:16:57.102055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 262214 count 1 nextOffset 13 batches 1
2025-11-26T14:16:57.102061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.102063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.102066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.102671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00000? size 262204 WTime 1764166617100
2025-11-26T14:16:57.102702Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.102713Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 262204
2025-11-26T14:16:57.105507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.105523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.105540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.105547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.105554Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-26T14:16:57.105578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.105581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.105583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.105586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.105587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.105592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.105602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.105698Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.105783Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    WriteTimestampMS: 1764166617100
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.106642Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.106769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768484:3397] destroyed
2025-11-26T14:16:57.107046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768488:3399], now have 1 active actors on pipe
2025-11-26T14:16:57.107210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.107224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.107245Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c7691ace-a0d7da63-f055091a-a2665aea_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.107251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.107255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.107257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.107259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.107261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.107266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.107268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.107270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.107279Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.107289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.107420Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c7691ace-a0d7da63-f055091a-a2665aea_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c7691ace-a0d7da63-f055091a-a2665aea_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.109798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768488:3399] destroyed
2025-11-26T14:16:57.109863Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.110518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768491:3400], now have 1 active actors on pipe
2025-11-26T14:16:57.110602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.110606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.110622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.110628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.110638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.110641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.110648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.110650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.110653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.110674Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-26T14:16:57.110698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 262214 count 1 nextOffset 14 batches 1
2025-11-26T14:16:57.110703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.110705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.110707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.110807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00000? size 262204 WTime 1764166617109
2025-11-26T14:16:57.110845Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.110855Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 262204
2025-11-26T14:16:57.113329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.113354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.113371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.113377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.113384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-26T14:16:57.113410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.113413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.113414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.113417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.113419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.113424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.113431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.113531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.113618Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    WriteTimestampMS: 1764166617109
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.114394Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.114912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768491:3400] destroyed
2025-11-26T14:16:57.114922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768495:3402], now have 1 active actors on pipe
2025-11-26T14:16:57.114929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.114934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.114956Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|569ca971-a9a9071c-47f56726-f63053b9_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.114962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.114965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.114967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.114969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.114971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.114976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.114978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.114979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.114987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.114995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.115138Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|569ca971-a9a9071c-47f56726-f63053b9_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|569ca971-a9a9071c-47f56726-f63053b9_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.116671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768495:3402] destroyed
2025-11-26T14:16:57.117553Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.118195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768498:3403], now have 1 active actors on pipe
2025-11-26T14:16:57.118273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.118277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.118295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.118301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.118312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.118314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.118322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.118323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.118325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.118347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-26T14:16:57.118371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 262214 count 1 nextOffset 15 batches 1
2025-11-26T14:16:57.118376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.118378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.118380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.118495Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00000? size 262204 WTime 1764166617117
2025-11-26T14:16:57.118525Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.118535Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 262204
2025-11-26T14:16:57.121063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.121076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.121090Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.121098Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.121107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-26T14:16:57.121132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.121135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.121137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.121139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.121140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.121145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.121151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.121251Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    WriteTimestampMS: 1764166617117
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.121336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.121943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768498:3403] destroyed
2025-11-26T14:16:57.122271Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.122790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768503:3406], now have 1 active actors on pipe
2025-11-26T14:16:57.122801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.122805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.122826Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|56413865-ad806977-6657520f-3cfbf3df_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.122833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.122836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.122839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.122840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.122842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.122848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.122850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.122853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.122861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.122868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.122991Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|56413865-ad806977-6657520f-3cfbf3df_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|56413865-ad806977-6657520f-3cfbf3df_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.125262Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.125655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768503:3406] destroyed
2025-11-26T14:16:57.125664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768506:3407], now have 1 active actors on pipe
2025-11-26T14:16:57.126205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.126212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.126231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.126237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.126247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.126249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.126256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.126258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.126260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.126280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-26T14:16:57.126306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 262214 count 1 nextOffset 16 batches 1
2025-11-26T14:16:57.126311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.126313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.126316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.126420Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00000? size 262204 WTime 1764166617123
2025-11-26T14:16:57.126451Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.126461Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 262204
2025-11-26T14:16:57.128793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.128807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.128823Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.128831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.128839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-26T14:16:57.128866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.128868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.128869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.128872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.128873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.128879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.128887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.128989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.129080Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.129438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768506:3407] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    WriteTimestampMS: 1764166617123
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.134061Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.140734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768510:3409], now have 1 active actors on pipe
2025-11-26T14:16:57.140993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.141007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.141045Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|31288f1b-cda144fc-eef578ea-a4e376a0_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.141058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.141062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.141069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.141071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.141075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.141087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.141089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.141092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.141111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.141130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.141267Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|31288f1b-cda144fc-eef578ea-a4e376a0_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|31288f1b-cda144fc-eef578ea-a4e376a0_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.144115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768510:3409] destroyed
2025-11-26T14:16:57.144221Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.144903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768513:3410], now have 1 active actors on pipe
2025-11-26T14:16:57.145178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.145189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.145214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.145226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.145242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.145245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.145256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.145258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.145263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.145293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-26T14:16:57.145323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 262214 count 1 nextOffset 17 batches 1
2025-11-26T14:16:57.145329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.145332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.145334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.145451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00000? size 262204 WTime 1764166617144
2025-11-26T14:16:57.145486Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.145498Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 262204
2025-11-26T14:16:57.151563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.151584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.151610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.151621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.151631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-26T14:16:57.151668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.151673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.151675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.151677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.151679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.151689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.151699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.151802Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.151904Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    WriteTimestampMS: 1764166617144
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.153122Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.156467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768513:3410] destroyed
2025-11-26T14:16:57.156482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768518:3413], now have 1 active actors on pipe
2025-11-26T14:16:57.156638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.156647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.156682Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7a065419-f1efb4c2-b1021a3b-b658944f_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.156693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.156697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.156703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.156705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.156710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.156721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.156725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.156727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.156746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.156760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.156890Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.157262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768518:3413] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7a065419-f1efb4c2-b1021a3b-b658944f_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7a065419-f1efb4c2-b1021a3b-b658944f_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.165506Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.166136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768521:3414], now have 1 active actors on pipe
2025-11-26T14:16:57.166600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.166612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.166637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.166649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.166668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.166672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.166684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.166686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.166691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.166723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-26T14:16:57.166761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 262214 count 1 nextOffset 18 batches 1
2025-11-26T14:16:57.166767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.166773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.166776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.166889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00000? size 262204 WTime 1764166617165
2025-11-26T14:16:57.166924Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.166935Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 262204
2025-11-26T14:16:57.173712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.173734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.173757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.173768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.173778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-26T14:16:57.173814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.173817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.173820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.173822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.173824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.173832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.173843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.173961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.174045Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.174606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768521:3414] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    WriteTimestampMS: 1764166617165
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.177246Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.179523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768525:3416], now have 1 active actors on pipe
2025-11-26T14:16:57.184563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.184580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.184623Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c137871d-9e4d6ddd-5cb19cf8-c49bb84f_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.184637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.184642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.184648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.184650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.184656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.184672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.184675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.184678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.184697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.184717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.184927Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.185315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768525:3416] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c137871d-9e4d6ddd-5cb19cf8-c49bb84f_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c137871d-9e4d6ddd-5cb19cf8-c49bb84f_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.193406Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.199513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.199527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.199530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.199536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.199539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.204615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768529:3418], now have 1 active actors on pipe
2025-11-26T14:16:57.205209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.205223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.205250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.205266Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.205285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.205290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.205304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.205307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.205313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.205347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-26T14:16:57.205389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 262214 count 1 nextOffset 19 batches 1
2025-11-26T14:16:57.205397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.205400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.205403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.205533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00000? size 262204 WTime 1764166617203
2025-11-26T14:16:57.205574Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.205586Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 262204
2025-11-26T14:16:57.208275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.208295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.208357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.208922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.208948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.208961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.208973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-26T14:16:57.209030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.209032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.209034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.209036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.209038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.209048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.209061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.209217Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.209755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768529:3418] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    WriteTimestampMS: 1764166617203
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.213188Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.216334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768533:3420], now have 1 active actors on pipe
2025-11-26T14:16:57.216364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.216372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.216418Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|121735e-ec9fa2fc-574e3887-9c6c8a91_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.216430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.216435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.216443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.216445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.216450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.216461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.216464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.216468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.216488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.216505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.216691Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.217205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768533:3420] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|121735e-ec9fa2fc-574e3887-9c6c8a91_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|121735e-ec9fa2fc-574e3887-9c6c8a91_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.225300Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.236150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768536:3421], now have 1 active actors on pipe
2025-11-26T14:16:57.236559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.236572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.236598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.236613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.236636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.236641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.236653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.236655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.236661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.236692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-26T14:16:57.236722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 262214 count 1 nextOffset 20 batches 1
2025-11-26T14:16:57.236729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.236731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.236734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.236862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00000? size 262204 WTime 1764166617236
2025-11-26T14:16:57.236898Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.236910Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 262204
2025-11-26T14:16:57.240955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.240974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.240994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.241004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.241012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-26T14:16:57.241043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.241046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.241049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.241051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.241053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.241060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.241068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.241158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.241224Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.241575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768536:3421] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    WriteTimestampMS: 1764166617236
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.245947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768540:3423], now have 1 active actors on pipe
2025-11-26T14:16:57.246114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.246125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.246166Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|222950b0-79035ca0-b0d67e05-52ad68d0_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.246179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.246185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.246191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.246194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.246198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.246210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.246213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.246219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.246239Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.246257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.246881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768540:3423] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|222950b0-79035ca0-b0d67e05-52ad68d0_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|222950b0-79035ca0-b0d67e05-52ad68d0_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.245384Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.246417Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.249349Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.249925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768543:3424], now have 1 active actors on pipe
2025-11-26T14:16:57.250411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.250423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.250448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.250460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.250477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.250482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.250494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.250497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.250502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.250535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-26T14:16:57.250569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 262214 count 1 nextOffset 21 batches 1
2025-11-26T14:16:57.250576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.250592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.250597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.250721Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00000? size 262204 WTime 1764166617245
2025-11-26T14:16:57.250758Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.250772Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 262204
2025-11-26T14:16:57.268842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.268874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.268903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.268917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.268931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-26T14:16:57.268976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.268980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.268982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.268986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.268988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.269002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.269019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.269158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.269251Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    WriteTimestampMS: 1764166617245
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 22
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.270473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768543:3424] destroyed
2025-11-26T14:16:57.284600Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.285441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768560:3433], now have 1 active actors on pipe
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ca1244eb-911f1821-a096cee7-a70c72e8_21"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ca1244eb-911f1821-a096cee7-a70c72e8_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.285809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.285820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.285858Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ca1244eb-911f1821-a096cee7-a70c72e8_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.285872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.285876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.285882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.285884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.285889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.285902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.285905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.285908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.285927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.285945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.291743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768560:3433] destroyed
2025-11-26T14:16:57.291292Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.299265Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.299950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768564:3435], now have 1 active actors on pipe
2025-11-26T14:16:57.300199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.300206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.300210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.300212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.300215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.300574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.300585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.300616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.300625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.300639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.300643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.300653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.300657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.300661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.300695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-26T14:16:57.300728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 262214 count 1 nextOffset 22 batches 1
2025-11-26T14:16:57.300735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.300738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.300741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.300856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00000? size 262204 WTime 1764166617300
2025-11-26T14:16:57.300894Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.300906Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 262204
2025-11-26T14:16:57.314314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.314348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.314373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.314386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.314397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-26T14:16:57.314436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.314440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.314442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.314445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.314449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.314459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.314472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.314621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.314916Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    WriteTimestampMS: 1764166617300
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.316104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768564:3435] destroyed
2025-11-26T14:16:57.316217Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.316972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768573:3438], now have 1 active actors on pipe
2025-11-26T14:16:57.316987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.316994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.317024Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|59dff8cd-a76da570-f1477b95-8bd93d41_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.317036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.317040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.317044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.317047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.317051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.317061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.317064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.317066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.317082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.317096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.317237Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|59dff8cd-a76da570-f1477b95-8bd93d41_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|59dff8cd-a76da570-f1477b95-8bd93d41_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.319994Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.320216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768573:3438] destroyed
2025-11-26T14:16:57.322814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768590:3442], now have 1 active actors on pipe
2025-11-26T14:16:57.323309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.323321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.323343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.323354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.323370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.323374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.323384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.323386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.323390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.323421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-26T14:16:57.323459Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 262214 count 1 nextOffset 23 batches 1
2025-11-26T14:16:57.323466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.323468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.323470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.323592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00000? size 262204 WTime 1764166617320
2025-11-26T14:16:57.323630Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.323641Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 262204
2025-11-26T14:16:57.326239Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.326259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.326279Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.326290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.326299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-26T14:16:57.326340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.326344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.326346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.326348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.326350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.326359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.326370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.326492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.326571Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    WriteTimestampMS: 1764166617320
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.327548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768590:3442] destroyed
2025-11-26T14:16:57.332556Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.336465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768595:3447], now have 1 active actors on pipe
2025-11-26T14:16:57.336722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.336732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.336768Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c984ca1f-bb81d867-a5ede2d5-507fc1a3_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.336780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.336787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.336794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.336796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.336800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.336811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.336813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.336816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.336834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.336852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.336994Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c984ca1f-bb81d867-a5ede2d5-507fc1a3_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c984ca1f-bb81d867-a5ede2d5-507fc1a3_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.339388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768595:3447] destroyed
2025-11-26T14:16:57.341803Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.342559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768598:3448], now have 1 active actors on pipe
2025-11-26T14:16:57.342980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.342991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.343023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.343034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.343052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.343057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.343066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.343069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.343073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.343104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-26T14:16:57.343139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 262214 count 1 nextOffset 24 batches 1
2025-11-26T14:16:57.343146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.343149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.343153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.343285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00000? size 262204 WTime 1764166617342
2025-11-26T14:16:57.343323Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.343335Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 262204
2025-11-26T14:16:57.350516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.350538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.350560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.350590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.350600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-26T14:16:57.350638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.350643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.350646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.350649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.350651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.350663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.350675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.350863Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-26T14:16:57.350800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    WriteTimestampMS: 1764166617342
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.351571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768598:3448] destroyed
2025-11-26T14:16:57.351921Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.352600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768611:3453], now have 1 active actors on pipe
2025-11-26T14:16:57.352614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.352621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.352652Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|907e91bb-27ec381a-81dcdfe-4f8779a8_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.352662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.352666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.352670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.352673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.352677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.352686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.352689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.352692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.352705Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.352716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.352845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|907e91bb-27ec381a-81dcdfe-4f8779a8_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:57.353175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768611:3453] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|907e91bb-27ec381a-81dcdfe-4f8779a8_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.355279Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.356115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768614:3454], now have 1 active actors on pipe
2025-11-26T14:16:57.356200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.356205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.356225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.356233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.356247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.356250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.356274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.356277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.356281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.356312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-26T14:16:57.356344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 262214 count 1 nextOffset 25 batches 1
2025-11-26T14:16:57.356351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.356354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.356359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.356480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00000? size 262204 WTime 1764166617355
2025-11-26T14:16:57.356513Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.356528Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 262204
2025-11-26T14:16:57.358616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.358633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.358651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.358659Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.358668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-26T14:16:57.358698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.358701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.358703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.358705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.358707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.358714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.358723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.358821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.358916Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.359213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768614:3454] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    WriteTimestampMS: 1764166617355
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 0
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.361277Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.361818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768618:3456], now have 1 active actors on pipe
2025-11-26T14:16:57.361832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.361837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.361865Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|dac1d663-73e85047-ed7da3bc-135cfb0a_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.361873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.361877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.361881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.361884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.361887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.361895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.361898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.361900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.361911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.361923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.362063Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|dac1d663-73e85047-ed7da3bc-135cfb0a_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|dac1d663-73e85047-ed7da3bc-135cfb0a_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.364592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768618:3456] destroyed
2025-11-26T14:16:57.364660Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.365366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768621:3457], now have 1 active actors on pipe
2025-11-26T14:16:57.365442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.365448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.365466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.365473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.365485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.365488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.365496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.365498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.365501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.365524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-26T14:16:57.365551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 262214 count 1 nextOffset 26 batches 1
2025-11-26T14:16:57.365556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.365558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.365560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.365673Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00000? size 262204 WTime 1764166617365
2025-11-26T14:16:57.365705Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.365718Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 262204
2025-11-26T14:16:57.367787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.367801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.367818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.367825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.367832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-26T14:16:57.367861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.367863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.367866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.367868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.367872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.367878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.367885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.367983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.368079Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    WriteTimestampMS: 1764166617365
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.368510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768621:3457] destroyed
2025-11-26T14:16:57.368860Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.369254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768625:3459], now have 1 active actors on pipe
2025-11-26T14:16:57.369363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.369374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.369402Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|519e01b1-2bc336c2-6be6c3e7-65f99c7d_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.369414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.369416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.369419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.369421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.369424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.369430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.369432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.369435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.369447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.369456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.369509Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|519e01b1-2bc336c2-6be6c3e7-65f99c7d_26"
    Status: Active
    SeqNo: 0
  }
}
2025-11-26T14:16:57.369702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768625:3459] destroyed
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|519e01b1-2bc336c2-6be6c3e7-65f99c7d_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.371559Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.372129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768628:3460], now have 1 active actors on pipe
2025-11-26T14:16:57.372568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.372577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.372595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.372603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.372614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.372617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.372625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.372628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.372631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.372652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-26T14:16:57.372682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 262214 count 1 nextOffset 27 batches 1
2025-11-26T14:16:57.372687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.372690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.372693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.372809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00000? size 262204 WTime 1764166617372
2025-11-26T14:16:57.372850Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.372867Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 262204
2025-11-26T14:16:57.374815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.374836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.374854Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.374861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.374868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-26T14:16:57.374895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.374897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.374899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.374901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.374903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.374909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.374918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.375016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.375081Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    WriteTimestampMS: 1764166617372
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-26T14:16:57.375463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768628:3460] destroyed
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.375947Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.376470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768632:3462], now have 1 active actors on pipe
2025-11-26T14:16:57.376480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.376484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.376507Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2c956e03-137c8672-19cdf30c-8e716e87_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.376514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.376518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.376521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.376523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.376526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.376532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.376534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.376537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.376546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.376554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.376677Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.376989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768632:3462] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2c956e03-137c8672-19cdf30c-8e716e87_27"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2c956e03-137c8672-19cdf30c-8e716e87_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.379117Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.379483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768635:3463], now have 1 active actors on pipe
2025-11-26T14:16:57.380791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.380807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.380828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.380837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.380854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.380863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.380871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.380877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.380881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.380906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-26T14:16:57.380962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 262214 count 1 nextOffset 28 batches 1
2025-11-26T14:16:57.380973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.380975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.380978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.381102Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00000? size 262204 WTime 1764166617379
2025-11-26T14:16:57.381148Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.381165Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 262204
2025-11-26T14:16:57.387439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.387470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.387492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.387503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.387518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-26T14:16:57.387527Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.387555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.387559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.387562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.387566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.387568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.387577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.387588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.387675Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    WriteTimestampMS: 1764166617379
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.388451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768635:3463] destroyed
2025-11-26T14:16:57.392330Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.396586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768639:3465], now have 1 active actors on pipe
2025-11-26T14:16:57.396816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.396826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.396873Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1edfc9fa-74c4a760-f6183072-c6d4ac6_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.396883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.396887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.396891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.396894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.396898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.396907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.396910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.396913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.396931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.396950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.397257Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1edfc9fa-74c4a760-f6183072-c6d4ac6_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1edfc9fa-74c4a760-f6183072-c6d4ac6_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.399871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768639:3465] destroyed
2025-11-26T14:16:57.400308Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.400888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768642:3466], now have 1 active actors on pipe
2025-11-26T14:16:57.401367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.401379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.401404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.401415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.401431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.401435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.401445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.401448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.401452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.401485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-26T14:16:57.401522Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 262214 count 1 nextOffset 29 batches 1
2025-11-26T14:16:57.401529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.401532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.401535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.401674Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00000? size 262204 WTime 1764166617399
2025-11-26T14:16:57.401712Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.401724Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 262204
2025-11-26T14:16:57.402946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.408514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.408539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.408558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.408561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.408572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.408582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-26T14:16:57.408618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.408625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.408627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.408636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.408637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.408645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.408655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.408734Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    WriteTimestampMS: 1764166617399
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.409999Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.412525Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|acb43007-aeb283ad-788ca7c5-459af6be_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|acb43007-aeb283ad-788ca7c5-459af6be_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.412299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768642:3466] destroyed
2025-11-26T14:16:57.412321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768646:3468], now have 1 active actors on pipe
2025-11-26T14:16:57.412339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.412345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.412384Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|acb43007-aeb283ad-788ca7c5-459af6be_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.412394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.412400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.412405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.412408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.412412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.412422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.412425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.412427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.412444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.412461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.412832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768646:3468] destroyed
2025-11-26T14:16:57.415553Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.416034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768649:3469], now have 1 active actors on pipe
2025-11-26T14:16:57.416505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.416516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.416540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.416549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.416563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.416567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.416577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.416580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.416584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.416612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-26T14:16:57.416641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 262214 count 1 nextOffset 30 batches 1
2025-11-26T14:16:57.416656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.416658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.416660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.416772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00000? size 262204 WTime 1764166617416
2025-11-26T14:16:57.416806Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.416818Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 262204
2025-11-26T14:16:57.419124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.419142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.419160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.419170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.419178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-26T14:16:57.419205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.419208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.419209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.419211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.419213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.419219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.419227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.419314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.419388Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    WriteTimestampMS: 1764166617416
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.419852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768649:3469] destroyed
2025-11-26T14:16:57.420392Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.420907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768653:3471], now have 1 active actors on pipe
2025-11-26T14:16:57.421201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.421209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.421235Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a8f81d9c-a406fd87-ba73a786-9d92c02c_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.421243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.421246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.421250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.421252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.421256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.421263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.421265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.421267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.421277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.421286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.421394Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a8f81d9c-a406fd87-ba73a786-9d92c02c_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a8f81d9c-a406fd87-ba73a786-9d92c02c_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.422701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768653:3471] destroyed
2025-11-26T14:16:57.423625Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.424083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768656:3472], now have 1 active actors on pipe
2025-11-26T14:16:57.424503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.424512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.424532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.424540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.424551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.424554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.424567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.424568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.424572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.424600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-26T14:16:57.424630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 262214 count 1 nextOffset 31 batches 1
2025-11-26T14:16:57.424635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.424637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.424639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.424763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00000? size 262204 WTime 1764166617424
2025-11-26T14:16:57.424798Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.424809Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 262204
2025-11-26T14:16:57.433300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.433321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.433345Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.433356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.433366Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-26T14:16:57.433405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.433408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.433410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.433412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.433414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.433423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.433435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.433557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.433622Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-26T14:16:57.434103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768656:3472] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    WriteTimestampMS: 1764166617424
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.436343Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.436950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768660:3474], now have 1 active actors on pipe
2025-11-26T14:16:57.436966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.436980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.437009Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9aabaa2a-beaf4dac-986d39a0-817acd86_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.437019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.437025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.437031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.437033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.437036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.437054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.437057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.437059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.437074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.437089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.437345Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9aabaa2a-beaf4dac-986d39a0-817acd86_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9aabaa2a-beaf4dac-986d39a0-817acd86_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.440616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768660:3474] destroyed
2025-11-26T14:16:57.440700Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.441685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768663:3475], now have 1 active actors on pipe
2025-11-26T14:16:57.441777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.441783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.441805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.441814Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.441826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.441830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.441840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.441842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.441846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.441871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-26T14:16:57.441901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 0 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-26T14:16:57.441906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.441909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.441911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.442018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000? size 262204 WTime 1764166617441
2025-11-26T14:16:57.442048Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.442058Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-26T14:16:57.445896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.445914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.445933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.445947Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.445955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-26T14:16:57.445985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.445988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.445990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.445992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.445994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.446003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00000? (262204)
2025-11-26T14:16:57.446091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-26T14:16:57.446102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.446119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-26T14:16:57.446121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-26T14:16:57.446130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-26T14:16:57.446140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-26T14:16:57.446153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-26T14:16:57.446159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-26T14:16:57.446179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-26T14:16:57.446196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-26T14:16:57.446202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-26T14:16:57.446208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-26T14:16:57.446255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-26T14:16:57.446269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-26T14:16:57.446275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-26T14:16:57.446281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-26T14:16:57.446287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-26T14:16:57.446293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-26T14:16:57.446301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-26T14:16:57.446308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-26T14:16:57.446370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-26T14:16:57.446377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-26T14:16:57.446384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-26T14:16:57.446390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-26T14:16:57.446398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-26T14:16:57.446404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-26T14:16:57.446411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-26T14:16:57.446417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-26T14:16:57.446424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-26T14:16:57.446431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-26T14:16:57.446437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-26T14:16:57.446444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-26T14:16:57.446450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-26T14:16:57.446457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-26T14:16:57.446465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-26T14:16:57.446479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-26T14:16:57.446489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 1 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:57.446492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:57.446495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:56.000000Z
2025-11-26T14:16:57.446498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:57.446607Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-26T14:16:57.446650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-26T14:16:57.446658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-26T14:16:57.446661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.446665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00000?, parameters.CurOffset=0
2025-11-26T14:16:57.446764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/1
2025-11-26T14:16:57.446772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-26T14:16:57.446793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-26T14:16:57.446796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-26T14:16:57.446800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-26T14:16:57.446801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.446803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00000?, parameters.CurOffset=1
2025-11-26T14:16:57.447237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/1
2025-11-26T14:16:57.447240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-26T14:16:57.448369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 524387 count 2 nextOffset 2 batches 1
2025-11-26T14:16:57.448375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-26T14:16:57.448383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-26T14:16:57.448386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.448392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00000?, parameters.CurOffset=2
2025-11-26T14:16:57.448701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/1
2025-11-26T14:16:57.448706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-26T14:16:57.448721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 786601 count 3 nextOffset 3 batches 2
2025-11-26T14:16:57.448723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-26T14:16:57.448726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-26T14:16:57.448728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.448730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00000?, parameters.CurOffset=3
2025-11-26T14:16:57.448804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/1
2025-11-26T14:16:57.448808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-26T14:16:57.449259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.449277Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449320Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449331Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449413Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.449436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:57.450686Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.450155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048774 count 4 nextOffset 4 batches 2
2025-11-26T14:16:57.450166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-26T14:16:57.450171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-26T14:16:57.450173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.450176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00000?, parameters.CurOffset=4
2025-11-26T14:16:57.450258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/1
2025-11-26T14:16:57.450262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-26T14:16:57.450275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1310988 count 5 nextOffset 5 batches 3
2025-11-26T14:16:57.450277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-26T14:16:57.450280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-26T14:16:57.450282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.450293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00000?, parameters.CurOffset=5
2025-11-26T14:16:57.450713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/1
2025-11-26T14:16:57.450717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-26T14:16:57.451728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1573161 count 6 nextOffset 6 batches 3
2025-11-26T14:16:57.451741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-26T14:16:57.451745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-26T14:16:57.451747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.451750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00000?, parameters.CurOffset=6
2025-11-26T14:16:57.451810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/1
2025-11-26T14:16:57.451819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-26T14:16:57.451829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1835375 count 7 nextOffset 7 batches 4
2025-11-26T14:16:57.451832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-26T14:16:57.451835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-26T14:16:57.451836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.451839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00000?, parameters.CurOffset=7
2025-11-26T14:16:57.451893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/1
2025-11-26T14:16:57.451901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-26T14:16:57.452766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097548 count 8 nextOffset 8 batches 4
2025-11-26T14:16:57.452777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-26T14:16:57.452782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-26T14:16:57.452784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.452788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00000?, parameters.CurOffset=8
2025-11-26T14:16:57.452851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/1
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    WriteTimestampMS: 1764166617441
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.453240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-26T14:16:57.453259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2359762 count 9 nextOffset 9 batches 5
2025-11-26T14:16:57.453262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-26T14:16:57.453265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-26T14:16:57.453267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.453269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00000?, parameters.CurOffset=9
2025-11-26T14:16:57.453363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/1
2025-11-26T14:16:57.453371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-26T14:16:57.453770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2621935 count 10 nextOffset 10 batches 5
2025-11-26T14:16:57.453779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-26T14:16:57.453783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-26T14:16:57.453784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.453787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00000?, parameters.CurOffset=10
2025-11-26T14:16:57.453847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/1
2025-11-26T14:16:57.453850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-26T14:16:57.453860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2884149 count 11 nextOffset 11 batches 6
2025-11-26T14:16:57.453862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-26T14:16:57.453866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-26T14:16:57.453867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.453869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00000?, parameters.CurOffset=11
2025-11-26T14:16:57.453926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/1
2025-11-26T14:16:57.453929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-26T14:16:57.454656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146322 count 12 nextOffset 12 batches 6
2025-11-26T14:16:57.454659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-26T14:16:57.454663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-26T14:16:57.454665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.454668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00000?, parameters.CurOffset=12
2025-11-26T14:16:57.454731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/1
2025-11-26T14:16:57.454734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-26T14:16:57.454754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3408536 count 13 nextOffset 13 batches 7
2025-11-26T14:16:57.454757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-26T14:16:57.454759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-26T14:16:57.454761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.454763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00000?, parameters.CurOffset=13
2025-11-26T14:16:57.454817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/1
2025-11-26T14:16:57.454820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-26T14:16:57.455717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3670709 count 14 nextOffset 14 batches 7
2025-11-26T14:16:57.455720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-26T14:16:57.455725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-26T14:16:57.455727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.455729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00000?, parameters.CurOffset=14
2025-11-26T14:16:57.455790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/1
2025-11-26T14:16:57.455794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-26T14:16:57.455801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3932923 count 15 nextOffset 15 batches 8
2025-11-26T14:16:57.455803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-26T14:16:57.455806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-26T14:16:57.455808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.455812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00000?, parameters.CurOffset=15
2025-11-26T14:16:57.455865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/1
2025-11-26T14:16:57.455867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-26T14:16:57.458189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195096 count 16 nextOffset 16 batches 8
2025-11-26T14:16:57.458199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-26T14:16:57.458207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-26T14:16:57.458209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.458213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00000?, parameters.CurOffset=16
2025-11-26T14:16:57.458286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/1
2025-11-26T14:16:57.458296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-26T14:16:57.458309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4457310 count 17 nextOffset 17 batches 9
2025-11-26T14:16:57.458311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-26T14:16:57.458315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-26T14:16:57.458316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.458318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00000?, parameters.CurOffset=17
2025-11-26T14:16:57.458374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/1
2025-11-26T14:16:57.458381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-26T14:16:57.459358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4719483 count 18 nextOffset 18 batches 9
2025-11-26T14:16:57.459367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-26T14:16:57.459372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-26T14:16:57.459373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.459376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00000?, parameters.CurOffset=18
2025-11-26T14:16:57.459443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/1
2025-11-26T14:16:57.459452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-26T14:16:57.459462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4981697 count 19 nextOffset 19 batches 10
2025-11-26T14:16:57.459464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-26T14:16:57.459467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-26T14:16:57.459469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.459471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00000?, parameters.CurOffset=19
2025-11-26T14:16:57.457167Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.459526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/1
2025-11-26T14:16:57.459537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-26T14:16:57.460501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243870 count 20 nextOffset 20 batches 10
2025-11-26T14:16:57.460505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-26T14:16:57.460510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-26T14:16:57.460512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.460514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00000?, parameters.CurOffset=20
2025-11-26T14:16:57.460577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/1
2025-11-26T14:16:57.460581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-26T14:16:57.460594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5506084 count 21 nextOffset 21 batches 11
2025-11-26T14:16:57.460596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-26T14:16:57.460598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-26T14:16:57.460600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.460602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00000?, parameters.CurOffset=21
2025-11-26T14:16:57.460655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/1
2025-11-26T14:16:57.460660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-26T14:16:57.461421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5768257 count 22 nextOffset 22 batches 11
2025-11-26T14:16:57.461423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-26T14:16:57.461428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-26T14:16:57.461429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.461432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00000?, parameters.CurOffset=22
2025-11-26T14:16:57.461493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/1
2025-11-26T14:16:57.461496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-26T14:16:57.461503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6030471 count 23 nextOffset 23 batches 12
2025-11-26T14:16:57.461505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-26T14:16:57.461508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-26T14:16:57.461509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.461512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00000?, parameters.CurOffset=23
2025-11-26T14:16:57.461563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/1
2025-11-26T14:16:57.461567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-26T14:16:57.462527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292644 count 24 nextOffset 24 batches 12
2025-11-26T14:16:57.462530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-26T14:16:57.462534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-26T14:16:57.462536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.462538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00000?, parameters.CurOffset=24
2025-11-26T14:16:57.462611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/1
2025-11-26T14:16:57.462614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-26T14:16:57.462634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6554858 count 25 nextOffset 25 batches 13
2025-11-26T14:16:57.462637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
2025-11-26T14:16:57.462641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-26T14:16:57.462644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.462647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00000?, parameters.CurOffset=25
2025-11-26T14:16:57.462698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/1
2025-11-26T14:16:57.462701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-26T14:16:57.464043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6817031 count 26 nextOffset 26 batches 13
2025-11-26T14:16:57.464056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-26T14:16:57.464062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-26T14:16:57.464064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.464066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00000?, parameters.CurOffset=26
2025-11-26T14:16:57.464126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/1
2025-11-26T14:16:57.464133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-26T14:16:57.464144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7079245 count 27 nextOffset 27 batches 14
2025-11-26T14:16:57.464148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-26T14:16:57.464151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-26T14:16:57.464153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.464156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00000?, parameters.CurOffset=27
2025-11-26T14:16:57.464209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/1
2025-11-26T14:16:57.464218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-26T14:16:57.464955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341418 count 28 nextOffset 28 batches 14
2025-11-26T14:16:57.464966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-26T14:16:57.464970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-26T14:16:57.464972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.464974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00000?, parameters.CurOffset=28
2025-11-26T14:16:57.465034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/1
2025-11-26T14:16:57.465037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-26T14:16:57.465047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7603632 count 29 nextOffset 29 batches 15
2025-11-26T14:16:57.465049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-26T14:16:57.465051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-26T14:16:57.465053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.465055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00000?, parameters.CurOffset=29
2025-11-26T14:16:57.465106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/1
2025-11-26T14:16:57.465108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-26T14:16:57.466072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7865805 count 30 nextOffset 30 batches 15
2025-11-26T14:16:57.466074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-26T14:16:57.466079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-26T14:16:57.466080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.466082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00000?, parameters.CurOffset=30
2025-11-26T14:16:57.466143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/1
2025-11-26T14:16:57.466146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-26T14:16:57.466153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 8128019 count 31 nextOffset 31 batches 16
2025-11-26T14:16:57.466155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-26T14:16:57.466159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-26T14:16:57.466161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:57.466164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00000?, parameters.CurOffset=31
2025-11-26T14:16:57.466217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/1
2025-11-26T14:16:57.466220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-26T14:16:57.478196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 31 partNo 0 result is x0000000000_00000000000000000000_00000_0000000031_00000 size 8128009
2025-11-26T14:16:57.478223Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000031_00000 new key d0000000000_00000000000000000000_00000_0000000031_00000 size 8128009 WTime 1764166617476
2025-11-26T14:16:57.478277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 0 FormedBlobsCount 1 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-26T14:16:57.478285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-26T14:16:57.478407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 0 endOffset 0 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000| size 262204 WTime 1764166617476
2025-11-26T14:16:57.478499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768663:3475] destroyed
2025-11-26T14:16:57.478510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768667:3477], now have 1 active actors on pipe
2025-11-26T14:16:57.478537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.478543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.478561Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.478576Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 31 size 8128009
2025-11-26T14:16:57.478600Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-26T14:16:57.478639Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|65f3aa7a-13b3b3ba-53d80307-4e8ce41b_32 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.478652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.478656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.478664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.478667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.478673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.478684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.478688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.478691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.478708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.478765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.479733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768667:3477] destroyed
2025-11-26T14:16:57.479200Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|65f3aa7a-13b3b3ba-53d80307-4e8ce41b_32"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|65f3aa7a-13b3b3ba-53d80307-4e8ce41b_32"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 32
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.485409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768674:3480], now have 1 active actors on pipe
2025-11-26T14:16:57.485518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.485525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.485546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 32 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.485557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.485575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.485580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.485591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.485593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.485597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.485624Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-26T14:16:57.485654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-26T14:16:57.485662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.485666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.485669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.485783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 32,1 HeadOffset 32 endOffset 32 curOffset 33 d0000000000_00000000000000000032_00000_0000000001_00000? size 262204 WTime 1764166617484
2025-11-26T14:16:57.485815Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.485826Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 0 count 1 size 262204
2025-11-26T14:16:57.484506Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.503551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 31 size 8128009 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.503565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.503578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 32 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.503587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.503709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.503713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.503716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.503736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.503746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.503757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 32, partNo: 0, Offset: 32 is stored on disk
2025-11-26T14:16:57.503778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.503781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.503783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.503786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.503788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.503840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.503848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.503858Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.503865Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00000?(+) to d0000000000_00000000000000000000_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503867Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00000?(+) to d0000000000_00000000000000000001_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503869Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00000?(+) to d0000000000_00000000000000000002_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503871Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00000?(+) to d0000000000_00000000000000000003_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503873Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00000?(+) to d0000000000_00000000000000000004_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503875Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00000?(+) to d0000000000_00000000000000000005_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503877Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00000?(+) to d0000000000_00000000000000000006_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503879Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00000?(+) to d0000000000_00000000000000000007_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503882Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00000?(+) to d0000000000_00000000000000000008_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503883Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00000?(+) to d0000000000_00000000000000000009_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503886Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00000?(+) to d0000000000_00000000000000000010_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503888Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00000?(+) to d0000000000_00000000000000000011_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503889Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00000?(+) to d0000000000_00000000000000000012_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503891Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00000?(+) to d0000000000_00000000000000000013_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503893Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00000?(+) to d0000000000_00000000000000000014_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503894Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00000?(+) to d0000000000_00000000000000000015_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503896Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00000?(+) to d0000000000_00000000000000000016_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503899Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00000?(+) to d0000000000_00000000000000000017_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503903Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00000?(+) to d0000000000_00000000000000000018_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503905Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00000?(+) to d0000000000_00000000000000000019_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503907Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00000?(+) to d0000000000_00000000000000000020_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503910Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00000?(+) to d0000000000_00000000000000000021_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503911Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00000?(+) to d0000000000_00000000000000000022_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503913Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00000?(+) to d0000000000_00000000000000000023_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503915Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00000?(+) to d0000000000_00000000000000000024_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503917Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00000?(+) to d0000000000_00000000000000000025_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503918Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00000?(+) to d0000000000_00000000000000000026_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503920Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00000?(+) to d0000000000_00000000000000000027_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503922Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00000?(+) to d0000000000_00000000000000000028_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503923Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00000?(+) to d0000000000_00000000000000000029_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503926Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00000?(+) to d0000000000_00000000000000000030_00000_0000000001_00000?(+)
2025-11-26T14:16:57.503927Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000?(+) to d0000000000_00000000000000000031_00000_0000000001_00000?(+)
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 32
    Offset: 32
    WriteTimestampMS: 1764166617484
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 15
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.504454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-26T14:16:57.504461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-26T14:16:57.504467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.504654Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.505748Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.506359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768674:3480] destroyed
2025-11-26T14:16:57.506370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768680:3484], now have 1 active actors on pipe
2025-11-26T14:16:57.506379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.506386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.506414Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|98f85495-77e6858d-638da11a-636ceb20_33 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.506422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.506975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.506983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.506989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.506995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507005Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507025Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507036Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507065Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 24 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507083Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 25 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507086Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 26 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 27 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 28 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507096Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 29 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507099Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 30 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.507118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.507285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507298Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507306Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507311Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507314Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507316Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507319Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507352Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.507370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.508583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.508591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.508606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.508615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.508619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.508626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.508627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.508631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.508639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.508642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.508645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.508660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.508667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.508677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.508840Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|98f85495-77e6858d-638da11a-636ceb20_33"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|98f85495-77e6858d-638da11a-636ceb20_33"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 33
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.511246Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.512070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768680:3484] destroyed
2025-11-26T14:16:57.512082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768684:3486], now have 1 active actors on pipe
2025-11-26T14:16:57.512157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.512163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.512184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 33 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.512195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.512207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.512211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.512222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.512224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.512241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.512287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-26T14:16:57.512326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 262214 count 1 nextOffset 34 batches 1
2025-11-26T14:16:57.512332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.512336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.512338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.512448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 33,1 HeadOffset 33 endOffset 33 curOffset 34 d0000000000_00000000000000000033_00000_0000000001_00000? size 262204 WTime 1764166617509
2025-11-26T14:16:57.512481Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.512492Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 33 partNo 0 count 1 size 262204
2025-11-26T14:16:57.514459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 33 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.514476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.514534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.518268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.518297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.518308Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.518321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 33, partNo: 0, Offset: 33 is stored on disk
2025-11-26T14:16:57.518358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.518362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.518367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.518370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.518372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.518381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.518395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.518609Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 33
    Offset: 33
    WriteTimestampMS: 1764166617509
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.519868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768684:3486] destroyed
2025-11-26T14:16:57.519931Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.520474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768688:3488], now have 1 active actors on pipe
2025-11-26T14:16:57.520619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.520628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.520658Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3b7ec0c3-623d75f7-10f079b9-a657fda5_34 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.520667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.520670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.520675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.520677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.520682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.520691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.520694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.520697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.520709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.520721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.520856Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3b7ec0c3-623d75f7-10f079b9-a657fda5_34"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3b7ec0c3-623d75f7-10f079b9-a657fda5_34"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 34
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.523252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768688:3488] destroyed
2025-11-26T14:16:57.523371Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.524180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768692:3490], now have 1 active actors on pipe
2025-11-26T14:16:57.524288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.524295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.524318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 34 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.524327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.524341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.524345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.524364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.524366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.524371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.524400Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-26T14:16:57.524433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 34 PartNo 0 PackedSize 262214 count 1 nextOffset 35 batches 1
2025-11-26T14:16:57.524441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.524443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.524445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.524555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 34,1 HeadOffset 34 endOffset 34 curOffset 35 d0000000000_00000000000000000034_00000_0000000001_00000? size 262204 WTime 1764166617524
2025-11-26T14:16:57.524586Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.524597Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 34 partNo 0 count 1 size 262204
2025-11-26T14:16:57.526794Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 34 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.526810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.526868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.528678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.528706Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.528717Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.528734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 34, partNo: 0, Offset: 34 is stored on disk
2025-11-26T14:16:57.528772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.528781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.528783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.528785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.528787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.528795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.528808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.528901Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.529446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768692:3490] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 34
    Offset: 34
    WriteTimestampMS: 1764166617524
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 4
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.530192Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.530743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768696:3492], now have 1 active actors on pipe
2025-11-26T14:16:57.530915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.530921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.530944Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|be94a931-3881071a-c33b764f-66c1ce9d_35 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.530950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.530952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.530956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.530957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.530959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.530965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.530967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.530969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.530977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.530986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.531139Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.531578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768696:3492] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|be94a931-3881071a-c33b764f-66c1ce9d_35"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|be94a931-3881071a-c33b764f-66c1ce9d_35"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 35
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.533880Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.534327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768699:3493], now have 1 active actors on pipe
2025-11-26T14:16:57.534855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.534864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.534883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 35 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.534890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.534900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.534903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.534911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.534913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.534916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.534938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-26T14:16:57.534968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 35 PartNo 0 PackedSize 262214 count 1 nextOffset 36 batches 1
2025-11-26T14:16:57.534975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.534977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.534981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.535103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 35,1 HeadOffset 35 endOffset 35 curOffset 36 d0000000000_00000000000000000035_00000_0000000001_00000? size 262204 WTime 1764166617532
2025-11-26T14:16:57.535132Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.535142Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 35 partNo 0 count 1 size 262204
2025-11-26T14:16:57.537635Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 35 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.537651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.537670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.537679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.537688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 35, partNo: 0, Offset: 35 is stored on disk
2025-11-26T14:16:57.537714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.537716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.537718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.537719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.537721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.537727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.537737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.537850Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.537913Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.538303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768699:3493] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 35
    Offset: 35
    WriteTimestampMS: 1764166617532
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.539026Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.539413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768703:3495], now have 1 active actors on pipe
2025-11-26T14:16:57.539607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.539614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.539637Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|877ad5bf-ed3ced25-d1b35c26-4aaad93_36 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.539644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.539646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.539649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.539663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.539666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.539673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.539675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.539677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.539688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.539697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.539863Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.540149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768703:3495] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|877ad5bf-ed3ced25-d1b35c26-4aaad93_36"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|877ad5bf-ed3ced25-d1b35c26-4aaad93_36"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 36
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.542418Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.542880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768706:3496], now have 1 active actors on pipe
2025-11-26T14:16:57.543369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.543377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.543406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 36 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.543413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.543424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.543427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.543435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.543437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.543441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.543466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-26T14:16:57.543499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 36 PartNo 0 PackedSize 262214 count 1 nextOffset 37 batches 1
2025-11-26T14:16:57.543506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.543509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.543511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.543634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 36,1 HeadOffset 36 endOffset 36 curOffset 37 d0000000000_00000000000000000036_00000_0000000001_00000? size 262204 WTime 1764166617543
2025-11-26T14:16:57.543666Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.543677Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 36 partNo 0 count 1 size 262204
2025-11-26T14:16:57.546015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 36 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.546031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.546052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.546060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.546068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 36, partNo: 0, Offset: 36 is stored on disk
2025-11-26T14:16:57.546101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.546104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.546106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.546108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.546111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.546121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.546129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.546255Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.546350Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.546915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768706:3496] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 36
    Offset: 36
    WriteTimestampMS: 1764166617543
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.547595Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.548072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768710:3498], now have 1 active actors on pipe
2025-11-26T14:16:57.548222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.548229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.548253Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4f16bf1d-9c8a71b6-5bc7e62f-d6ecf6dd_37 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.548271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.548274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.548278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.548280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.548283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.548290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.548292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.548294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.548305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.548316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.548462Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.548889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768710:3498] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4f16bf1d-9c8a71b6-5bc7e62f-d6ecf6dd_37"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4f16bf1d-9c8a71b6-5bc7e62f-d6ecf6dd_37"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 37
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.551222Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.551822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768713:3499], now have 1 active actors on pipe
2025-11-26T14:16:57.552239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.552247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.552278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 37 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.552287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.552301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.552306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.552315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.552317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.552320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.552360Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-26T14:16:57.552392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 262214 count 1 nextOffset 38 batches 1
2025-11-26T14:16:57.552398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.552400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.552403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.552840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 37,1 HeadOffset 37 endOffset 37 curOffset 38 d0000000000_00000000000000000037_00000_0000000001_00000? size 262204 WTime 1764166617551
2025-11-26T14:16:57.552874Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.552890Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 37 partNo 0 count 1 size 262204
2025-11-26T14:16:57.558962Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 37 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.558986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.559006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.559016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.559023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 37, partNo: 0, Offset: 37 is stored on disk
2025-11-26T14:16:57.559055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.559059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.559062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.559064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.559076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.559084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.559093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.559232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.559297Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 37
    Offset: 37
    WriteTimestampMS: 1764166617551
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.560434Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.561002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768713:3499] destroyed
2025-11-26T14:16:57.561015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768717:3501], now have 1 active actors on pipe
2025-11-26T14:16:57.561027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.561033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.561061Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b4ea331e-b0b152c2-b465dc3e-efd92c1b_38 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.561070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.561074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.561079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.561080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.561083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.561091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.561093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.561096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.561107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.561117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.561310Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b4ea331e-b0b152c2-b465dc3e-efd92c1b_38"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b4ea331e-b0b152c2-b465dc3e-efd92c1b_38"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 38
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.564016Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.564118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768717:3501] destroyed
2025-11-26T14:16:57.566784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768720:3502], now have 1 active actors on pipe
2025-11-26T14:16:57.566904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.566912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.566936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 38 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.566950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.566964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.566968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.566979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.566982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.566999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.567032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-26T14:16:57.567070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 38 PartNo 0 PackedSize 262214 count 1 nextOffset 39 batches 1
2025-11-26T14:16:57.567077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.567092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.567095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.567229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 38,1 HeadOffset 38 endOffset 38 curOffset 39 d0000000000_00000000000000000038_00000_0000000001_00000? size 262204 WTime 1764166617563
2025-11-26T14:16:57.567273Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.567287Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 38 partNo 0 count 1 size 262204
2025-11-26T14:16:57.573475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 38 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.573501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.573526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.573538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.573548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 38, partNo: 0, Offset: 38 is stored on disk
2025-11-26T14:16:57.573585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.573589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.573591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.573595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.573597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.573606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.573617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.573722Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.573794Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 38
    Offset: 38
    WriteTimestampMS: 1764166617563
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.575010Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.574993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768720:3502] destroyed
2025-11-26T14:16:57.575492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768724:3504], now have 1 active actors on pipe
2025-11-26T14:16:57.575633Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.575641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.575670Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|78888bd-dfa45245-e480dfd0-24b910ac_39 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.575680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.575683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.575687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.575689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.575692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.575703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.575705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.575707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.575719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.575730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.575870Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|78888bd-dfa45245-e480dfd0-24b910ac_39"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|78888bd-dfa45245-e480dfd0-24b910ac_39"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 39
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.578362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768724:3504] destroyed
2025-11-26T14:16:57.578487Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.578900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768727:3505], now have 1 active actors on pipe
2025-11-26T14:16:57.579322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.579331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.579357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 39 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.579365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.579380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.579383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.579407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.579408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.579412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.579442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-26T14:16:57.579475Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 39 PartNo 0 PackedSize 262214 count 1 nextOffset 40 batches 1
2025-11-26T14:16:57.579482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.579485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.579488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.579606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 39,1 HeadOffset 39 endOffset 39 curOffset 40 d0000000000_00000000000000000039_00000_0000000001_00000? size 262204 WTime 1764166617576
2025-11-26T14:16:57.579645Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.579657Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 39 partNo 0 count 1 size 262204
2025-11-26T14:16:57.582325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 39 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.582343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.582361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.582369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.582377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 39, partNo: 0, Offset: 39 is stored on disk
2025-11-26T14:16:57.582410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.582412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.582414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.582416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.582417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.582423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.582430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.582533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.582620Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.583029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768727:3505] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 39
    Offset: 39
    WriteTimestampMS: 1764166617576
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.583636Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.584200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768731:3507], now have 1 active actors on pipe
2025-11-26T14:16:57.584216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.584221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.584242Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8b6e0f1a-6342462f-81d4cebe-ec444b5a_40 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.584250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.584253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.584256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.584273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.584275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.584282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.584284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.584286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.584295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.584304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.584458Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.584695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768731:3507] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8b6e0f1a-6342462f-81d4cebe-ec444b5a_40"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8b6e0f1a-6342462f-81d4cebe-ec444b5a_40"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 40
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.588321Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.588895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768734:3508], now have 1 active actors on pipe
2025-11-26T14:16:57.589316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.589325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.589346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 40 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.589355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.589368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.589371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.589382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.589386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.589390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.589418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-26T14:16:57.589447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 40 PartNo 0 PackedSize 262214 count 1 nextOffset 41 batches 1
2025-11-26T14:16:57.589453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.589456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.589458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.589567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 40,1 HeadOffset 40 endOffset 40 curOffset 41 d0000000000_00000000000000000040_00000_0000000001_00000? size 262204 WTime 1764166617586
2025-11-26T14:16:57.589599Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.589610Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 40 partNo 0 count 1 size 262204
2025-11-26T14:16:57.597609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 40 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.597633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.597658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.597668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.597679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 40, partNo: 0, Offset: 40 is stored on disk
2025-11-26T14:16:57.597719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.597723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.597725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.597728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.597731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.597741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.597752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.597867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.597936Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 40
    Offset: 40
    WriteTimestampMS: 1764166617586
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.598888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768734:3508] destroyed
2025-11-26T14:16:57.600823Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.604519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768738:3510], now have 1 active actors on pipe
2025-11-26T14:16:57.608583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.608601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.608642Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|79600475-1bf60d8b-1a927458-22160f96_41 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.608655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.608659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.608665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.608670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.608674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.608687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.608690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.608693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.608712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.608731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.608886Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|79600475-1bf60d8b-1a927458-22160f96_41"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|79600475-1bf60d8b-1a927458-22160f96_41"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 41
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.610683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768738:3510] destroyed
2025-11-26T14:16:57.612524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.612534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.612537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.612541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.612544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.613254Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.613901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768741:3511], now have 1 active actors on pipe
2025-11-26T14:16:57.614373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.614382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.614401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 41 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.614410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.614421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.614424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.614433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.614435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.614439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.614463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-26T14:16:57.614492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 41 PartNo 0 PackedSize 262214 count 1 nextOffset 42 batches 1
2025-11-26T14:16:57.614498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.614501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.614504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.614633Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 41,1 HeadOffset 41 endOffset 41 curOffset 42 d0000000000_00000000000000000041_00000_0000000001_00000? size 262204 WTime 1764166617613
2025-11-26T14:16:57.614668Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.614679Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 41 partNo 0 count 1 size 262204
2025-11-26T14:16:57.616896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 41 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.616915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.616934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.616944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.616953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 41, partNo: 0, Offset: 41 is stored on disk
2025-11-26T14:16:57.616983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.616985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.616987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.616990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.616991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.616997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.617009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.617062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.617193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 41
    Offset: 41
    WriteTimestampMS: 1764166617613
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.618341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768741:3511] destroyed
2025-11-26T14:16:57.618406Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.619146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768745:3513], now have 1 active actors on pipe
2025-11-26T14:16:57.619158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.619164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.619190Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cf2e7012-1e69c318-a648789-b666b575_42 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.619199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.619202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.619206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.619208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.619211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.619220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.619222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.619224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.619235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.619246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.619360Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cf2e7012-1e69c318-a648789-b666b575_42"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cf2e7012-1e69c318-a648789-b666b575_42"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 42
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.621817Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.622604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768745:3513] destroyed
2025-11-26T14:16:57.622615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768748:3514], now have 1 active actors on pipe
2025-11-26T14:16:57.622693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.622698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.622719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 42 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.622729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.622742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.622745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.622755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.622756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.622760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.622787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-26T14:16:57.622816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 42 PartNo 0 PackedSize 262214 count 1 nextOffset 43 batches 1
2025-11-26T14:16:57.622822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.622824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.622826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.622950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 42,1 HeadOffset 42 endOffset 42 curOffset 43 d0000000000_00000000000000000042_00000_0000000001_00000? size 262204 WTime 1764166617620
2025-11-26T14:16:57.622980Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.622991Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 42 partNo 0 count 1 size 262204
2025-11-26T14:16:57.625191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 42 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.625207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.625226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.625233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.625241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 42, partNo: 0, Offset: 42 is stored on disk
2025-11-26T14:16:57.625267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.625269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.625271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.625273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.625274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.625281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.625289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.625381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.625446Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 42
    Offset: 42
    WriteTimestampMS: 1764166617620
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.626387Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.627030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768748:3514] destroyed
2025-11-26T14:16:57.627041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768752:3516], now have 1 active actors on pipe
2025-11-26T14:16:57.627049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.627055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.627079Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e01e06de-7b023922-705757a6-59434b19_43 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.627087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.627090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.627096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.627098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.627100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.627108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.627120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.627122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.627134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.627142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.627290Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e01e06de-7b023922-705757a6-59434b19_43"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e01e06de-7b023922-705757a6-59434b19_43"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 43
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.629648Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.630502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768752:3516] destroyed
2025-11-26T14:16:57.630513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768755:3517], now have 1 active actors on pipe
2025-11-26T14:16:57.630610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.630616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.630637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 43 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.630657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.630668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.630672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.630680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.630683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.630687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.630712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-26T14:16:57.630741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 43 PartNo 0 PackedSize 262214 count 1 nextOffset 44 batches 1
2025-11-26T14:16:57.630747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.630750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.630752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.630865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 43,1 HeadOffset 43 endOffset 43 curOffset 44 d0000000000_00000000000000000043_00000_0000000001_00000? size 262204 WTime 1764166617630
2025-11-26T14:16:57.630897Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.630908Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 43 partNo 0 count 1 size 262204
2025-11-26T14:16:57.633521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 43 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.633541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.633608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.637134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.637168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.637184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.637195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 43, partNo: 0, Offset: 43 is stored on disk
2025-11-26T14:16:57.637247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.637259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.637261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.637263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.637267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.637277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.637292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.640369Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 43
    Offset: 43
    WriteTimestampMS: 1764166617630
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.641678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768755:3517] destroyed
2025-11-26T14:16:57.648660Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.649264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768759:3519], now have 1 active actors on pipe
2025-11-26T14:16:57.652653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.652673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.652720Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ed003286-22e4bfb7-5ec9f62d-427cdf2e_44 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.652736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.652741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.652747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.652750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.652755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.652767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.652771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.652774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.652794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.652814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.652985Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.653399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768759:3519] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ed003286-22e4bfb7-5ec9f62d-427cdf2e_44"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ed003286-22e4bfb7-5ec9f62d-427cdf2e_44"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 44
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.656972Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.657560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768763:3521], now have 1 active actors on pipe
2025-11-26T14:16:57.658006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.658018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.658042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 44 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.658053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.658071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.658076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.658088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.658092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.658097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.658130Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-26T14:16:57.658165Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 44 PartNo 0 PackedSize 262214 count 1 nextOffset 45 batches 1
2025-11-26T14:16:57.658172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.658175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.658178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.658300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 44,1 HeadOffset 44 endOffset 44 curOffset 45 d0000000000_00000000000000000044_00000_0000000001_00000? size 262204 WTime 1764166617658
2025-11-26T14:16:57.658339Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.658352Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 44 partNo 0 count 1 size 262204
2025-11-26T14:16:57.663294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 44 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.663316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.663341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.663354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.663365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 44, partNo: 0, Offset: 44 is stored on disk
2025-11-26T14:16:57.663403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.663407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.663410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.663413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.663416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.663426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.663437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.663549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.663626Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 44
    Offset: 44
    WriteTimestampMS: 1764166617656
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.664959Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.666297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768763:3521] destroyed
2025-11-26T14:16:57.666311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768767:3523], now have 1 active actors on pipe
2025-11-26T14:16:57.666332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.666339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.666373Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b64f6369-d6713d9-ac06ac3e-ab1d5a9d_45 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.666384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.666390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.666395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.666397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.666400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.666415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.666418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.666421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.666435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.666450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.666617Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.667079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768767:3523] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b64f6369-d6713d9-ac06ac3e-ab1d5a9d_45"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b64f6369-d6713d9-ac06ac3e-ab1d5a9d_45"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 45
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.669461Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.670259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768770:3524], now have 1 active actors on pipe
2025-11-26T14:16:57.670340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.670347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.670370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 45 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.670380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.670394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.670398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.670409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.670411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.670416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.670463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-26T14:16:57.670496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 45 PartNo 0 PackedSize 262214 count 1 nextOffset 46 batches 1
2025-11-26T14:16:57.670503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.670506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.670509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.670641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 45,1 HeadOffset 45 endOffset 45 curOffset 46 d0000000000_00000000000000000045_00000_0000000001_00000? size 262204 WTime 1764166617670
2025-11-26T14:16:57.670681Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.670694Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 45 partNo 0 count 1 size 262204
2025-11-26T14:16:57.674194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 45 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.674215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.674238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.674248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.674258Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 45, partNo: 0, Offset: 45 is stored on disk
2025-11-26T14:16:57.674294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.674301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.674303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.674307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.674309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.674318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.674331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.674449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.674554Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.675186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768770:3524] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 45
    Offset: 45
    WriteTimestampMS: 1764166617670
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.675972Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.678544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768774:3526], now have 1 active actors on pipe
2025-11-26T14:16:57.678562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.678569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.678616Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5841b60a-c05b31ed-a1b47b55-f24ee01d_46 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.678628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.678631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.678636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.678639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.678642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.678653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.678656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.678659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.678675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.678691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.678840Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.679349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768774:3526] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5841b60a-c05b31ed-a1b47b55-f24ee01d_46"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5841b60a-c05b31ed-a1b47b55-f24ee01d_46"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 46
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.681761Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.682560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768777:3527], now have 1 active actors on pipe
2025-11-26T14:16:57.682664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.682671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.682696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 46 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.682708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.682725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.682729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.682744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.682747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.682751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.682796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-26T14:16:57.682833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 262214 count 1 nextOffset 47 batches 1
2025-11-26T14:16:57.682839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.682842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.682848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.682979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 46,1 HeadOffset 46 endOffset 46 curOffset 47 d0000000000_00000000000000000046_00000_0000000001_00000? size 262204 WTime 1764166617680
2025-11-26T14:16:57.683041Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.683055Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 46 partNo 0 count 1 size 262204
2025-11-26T14:16:57.685747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 46 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.685765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.685787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.685798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.685807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 46, partNo: 0, Offset: 46 is stored on disk
2025-11-26T14:16:57.685840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.685844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.685846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.685848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.685851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.685860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.685870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.685978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.686076Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.686651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768777:3527] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 46
    Offset: 46
    WriteTimestampMS: 1764166617680
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.687325Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.688061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768781:3529], now have 1 active actors on pipe
2025-11-26T14:16:57.688080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.688086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.688115Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3133a7f3-284a3e99-71c1c577-ea21409a_47 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.688126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.688130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.688134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.688136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.688140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.688150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.688152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.688155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.688170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.688184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.688335Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.688762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768781:3529] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3133a7f3-284a3e99-71c1c577-ea21409a_47"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3133a7f3-284a3e99-71c1c577-ea21409a_47"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 47
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.691136Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.691939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768784:3530], now have 1 active actors on pipe
2025-11-26T14:16:57.692035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.692040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.692058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 47 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.692066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.692076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.692081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.692088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.692090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.692093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.692116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-26T14:16:57.692158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 47 PartNo 0 PackedSize 262214 count 1 nextOffset 48 batches 1
2025-11-26T14:16:57.692163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.692166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.692168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.692306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 47,1 HeadOffset 47 endOffset 47 curOffset 48 d0000000000_00000000000000000047_00000_0000000001_00000? size 262204 WTime 1764166617691
2025-11-26T14:16:57.692344Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.692355Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 47 partNo 0 count 1 size 262204
2025-11-26T14:16:57.705209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 47 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.705238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.705265Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.705278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.705292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 47, partNo: 0, Offset: 47 is stored on disk
2025-11-26T14:16:57.705345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.705349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.705353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.705356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.705359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.705372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.705387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.705536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.705645Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.706265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768784:3530] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 47
    Offset: 47
    WriteTimestampMS: 1764166617691
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.707153Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.707826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768789:3533], now have 1 active actors on pipe
2025-11-26T14:16:57.707843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.707850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.707882Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7f3f437e-6f86f540-cd63ddb-26d7c663_48 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.707894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.707898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.707903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.707905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.707909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.707920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.707923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.707925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.707940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.707952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.708092Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.708532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768789:3533] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7f3f437e-6f86f540-cd63ddb-26d7c663_48"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7f3f437e-6f86f540-cd63ddb-26d7c663_48"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 48
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.710902Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.711734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768792:3534], now have 1 active actors on pipe
2025-11-26T14:16:57.711836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.711843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.711867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 48 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.711876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.711891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.711896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.711911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.711913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.711917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.711960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-26T14:16:57.711996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 48 PartNo 0 PackedSize 262214 count 1 nextOffset 49 batches 1
2025-11-26T14:16:57.712003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.712006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.712008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.712135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 48,1 HeadOffset 48 endOffset 48 curOffset 49 d0000000000_00000000000000000048_00000_0000000001_00000? size 262204 WTime 1764166617697
2025-11-26T14:16:57.712172Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.712185Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 48 partNo 0 count 1 size 262204
2025-11-26T14:16:57.718684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.724689Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 48 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.724713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.724741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.724754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.724773Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 48, partNo: 0, Offset: 48 is stored on disk
2025-11-26T14:16:57.724818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.724822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.724824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.724828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.724830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.724842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.724854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.724951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.725097Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 48
    Offset: 48
    WriteTimestampMS: 1764166617697
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 27
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.726419Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.726674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768792:3534] destroyed
2025-11-26T14:16:57.726990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768796:3536], now have 1 active actors on pipe
2025-11-26T14:16:57.727125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.727134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.727165Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ef5f1ebd-9c2ee23d-6d030e47-c41301a0_49 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.727177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.727182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.727187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.727189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.727193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.727204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.727207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.727210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.727227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.727241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.727374Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ef5f1ebd-9c2ee23d-6d030e47-c41301a0_49"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ef5f1ebd-9c2ee23d-6d030e47-c41301a0_49"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 49
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.730283Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.730451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768796:3536] destroyed
2025-11-26T14:16:57.730845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768799:3537], now have 1 active actors on pipe
2025-11-26T14:16:57.731292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.731304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.731332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 49 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.731342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.731357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.731361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.731372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.731374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.731378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.731408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-26T14:16:57.731439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 49 PartNo 0 PackedSize 262214 count 1 nextOffset 50 batches 1
2025-11-26T14:16:57.731445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.731449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.731452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.731578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 49,1 HeadOffset 49 endOffset 49 curOffset 50 d0000000000_00000000000000000049_00000_0000000001_00000? size 262204 WTime 1764166617728
2025-11-26T14:16:57.731614Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.731627Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 49 partNo 0 count 1 size 262204
2025-11-26T14:16:57.734874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 49 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.734905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.734930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.734942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.734952Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 49, partNo: 0, Offset: 49 is stored on disk
2025-11-26T14:16:57.734989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.734992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.734994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.734999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.735001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.735011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.735022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.735140Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.735216Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 49
    Offset: 49
    WriteTimestampMS: 1764166617728
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.736449Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.740330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768799:3537] destroyed
2025-11-26T14:16:57.740352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768803:3539], now have 1 active actors on pipe
2025-11-26T14:16:57.740370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.740377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.740418Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|40298258-3189a21-af0646c-72d1c68_50 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.740430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.740434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.740440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.740442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.740446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.740468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.740472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.740476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.740494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.740510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.740678Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|40298258-3189a21-af0646c-72d1c68_50"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|40298258-3189a21-af0646c-72d1c68_50"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 50
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.746874Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.746992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768803:3539] destroyed
2025-11-26T14:16:57.752824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768806:3540], now have 1 active actors on pipe
2025-11-26T14:16:57.752936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.752945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.752970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 50 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.752982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.752998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.753002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.753013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.753015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.753020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.753052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-26T14:16:57.753091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 50 PartNo 0 PackedSize 262214 count 1 nextOffset 51 batches 1
2025-11-26T14:16:57.753098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.753103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.753106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.753226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 50,1 HeadOffset 50 endOffset 50 curOffset 51 d0000000000_00000000000000000050_00000_0000000001_00000? size 262204 WTime 1764166617752
2025-11-26T14:16:57.753263Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.753276Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 50 partNo 0 count 1 size 262204
2025-11-26T14:16:57.756078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 50 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.756100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.756124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.756135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.756146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 50, partNo: 0, Offset: 50 is stored on disk
2025-11-26T14:16:57.756179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.756182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.756184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.756187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.756189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.756197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.756207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.756330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.756428Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 50
    Offset: 50
    WriteTimestampMS: 1764166617752
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.757572Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.758242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768806:3540] destroyed
2025-11-26T14:16:57.758255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768810:3542], now have 1 active actors on pipe
2025-11-26T14:16:57.758264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.758270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.758297Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8dad4e22-93e83ce-480d6ae7-15474170_51 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.758306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.758310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.758314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.758316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.758319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.758328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.758331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.758333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.758345Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.758356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.758492Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.758886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768810:3542] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8dad4e22-93e83ce-480d6ae7-15474170_51"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8dad4e22-93e83ce-480d6ae7-15474170_51"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 51
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.761391Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.762354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768813:3543], now have 1 active actors on pipe
2025-11-26T14:16:57.762459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.762466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.762490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 51 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.762498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.762512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.762516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.762527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.762529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.762532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.762561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-26T14:16:57.762607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 51 PartNo 0 PackedSize 262214 count 1 nextOffset 52 batches 1
2025-11-26T14:16:57.762613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.762617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.762619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.762737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 51,1 HeadOffset 51 endOffset 51 curOffset 52 d0000000000_00000000000000000051_00000_0000000001_00000? size 262204 WTime 1764166617760
2025-11-26T14:16:57.762770Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.762783Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 51 partNo 0 count 1 size 262204
2025-11-26T14:16:57.765306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 51 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.765323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.765342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.765350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.765361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 51, partNo: 0, Offset: 51 is stored on disk
2025-11-26T14:16:57.765395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.765399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.765401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.765402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.765404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.765411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.765421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.765533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.765622Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.766115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768813:3543] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 51
    Offset: 51
    WriteTimestampMS: 1764166617760
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.766748Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.767331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768817:3545], now have 1 active actors on pipe
2025-11-26T14:16:57.767343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.767347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.767370Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d319e3e4-dad6f302-57f434d6-e7a1b577_52 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.767377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.767379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.767384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.767386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.767389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.767396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.767399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.767401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.767411Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.767420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.767552Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.767899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768817:3545] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d319e3e4-dad6f302-57f434d6-e7a1b577_52"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d319e3e4-dad6f302-57f434d6-e7a1b577_52"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 52
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.770203Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.770946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768820:3546], now have 1 active actors on pipe
2025-11-26T14:16:57.771040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.771045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.771063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 52 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.771074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.771084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.771087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.771095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.771098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.771101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.771120Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-26T14:16:57.771145Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 52 PartNo 0 PackedSize 262214 count 1 nextOffset 53 batches 1
2025-11-26T14:16:57.771150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.771152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.771154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.771280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 52,1 HeadOffset 52 endOffset 52 curOffset 53 d0000000000_00000000000000000052_00000_0000000001_00000? size 262204 WTime 1764166617769
2025-11-26T14:16:57.771311Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.771322Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 52 partNo 0 count 1 size 262204
2025-11-26T14:16:57.777396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 52 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.777418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.777441Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.777452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.777465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 52, partNo: 0, Offset: 52 is stored on disk
2025-11-26T14:16:57.777503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.777507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.777509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.777512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.777514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.777525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.777536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.777652Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.777758Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.778407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768820:3546] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 52
    Offset: 52
    WriteTimestampMS: 1764166617769
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.779156Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.779821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768824:3548], now have 1 active actors on pipe
2025-11-26T14:16:57.779836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.779842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.779871Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fcd9361a-b3c7e9cc-dbe94d35-a2093fe6_53 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.779882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.779885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.779890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.779892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.779895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.779905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.779907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.779910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.779923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.779935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.780067Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.780473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768824:3548] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fcd9361a-b3c7e9cc-dbe94d35-a2093fe6_53"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fcd9361a-b3c7e9cc-dbe94d35-a2093fe6_53"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 53
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.782894Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.783682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768827:3549], now have 1 active actors on pipe
2025-11-26T14:16:57.783768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.783774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.783797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 53 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.783806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.783818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.783822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.783832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.783834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.783838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.783869Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-26T14:16:57.783905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 53 PartNo 0 PackedSize 262214 count 1 nextOffset 54 batches 1
2025-11-26T14:16:57.783924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.783927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.783929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.784047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 53,1 HeadOffset 53 endOffset 53 curOffset 54 d0000000000_00000000000000000053_00000_0000000001_00000? size 262204 WTime 1764166617780
2025-11-26T14:16:57.784081Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.784095Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 53 partNo 0 count 1 size 262204
2025-11-26T14:16:57.786552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 53 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.786592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.786618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.786628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.786637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 53, partNo: 0, Offset: 53 is stored on disk
2025-11-26T14:16:57.786665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.786668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.786671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.786673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.786675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.786682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.786692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.786797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.786895Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.787424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768827:3549] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 53
    Offset: 53
    WriteTimestampMS: 1764166617780
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.789888Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.790521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768831:3551], now have 1 active actors on pipe
2025-11-26T14:16:57.790542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.790548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.790592Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b9767432-a0f8a922-cf05373d-86094d0d_54 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.790601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.790607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.790612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.790614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.790618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.790627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.790631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.790634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.790648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.790661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.790800Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.791196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768831:3551] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b9767432-a0f8a922-cf05373d-86094d0d_54"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b9767432-a0f8a922-cf05373d-86094d0d_54"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 54
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.793617Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.794416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768834:3552], now have 1 active actors on pipe
2025-11-26T14:16:57.794506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.794511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.794532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 54 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.794541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.794552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.794557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.794567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.794570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.794574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.794630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-26T14:16:57.794662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 54 PartNo 0 PackedSize 262214 count 1 nextOffset 55 batches 1
2025-11-26T14:16:57.794669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.794672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.794675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.794796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 54,1 HeadOffset 54 endOffset 54 curOffset 55 d0000000000_00000000000000000054_00000_0000000001_00000? size 262204 WTime 1764166617792
2025-11-26T14:16:57.794830Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.794844Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 54 partNo 0 count 1 size 262204
2025-11-26T14:16:57.797645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 54 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.797663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.797684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.797693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.797702Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 54, partNo: 0, Offset: 54 is stored on disk
2025-11-26T14:16:57.797733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.797735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.797740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.797741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.797744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.797751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.797759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.797863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.797955Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.798449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768834:3552] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 54
    Offset: 54
    WriteTimestampMS: 1764166617792
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.800491Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.801459Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b83a3af1-2dfd82a9-31eaf4ae-ab86cb6b_55"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b83a3af1-2dfd82a9-31eaf4ae-ab86cb6b_55"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 55
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.801243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768838:3554], now have 1 active actors on pipe
2025-11-26T14:16:57.801259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.801265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.801297Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b83a3af1-2dfd82a9-31eaf4ae-ab86cb6b_55 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.801308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.801312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.801317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.801319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.801323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.801334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.801337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.801340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.801355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.801371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.801908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768838:3554] destroyed
2025-11-26T14:16:57.808151Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.808796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768841:3555], now have 1 active actors on pipe
2025-11-26T14:16:57.809289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.809300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.809327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 55 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.809341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.809361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.809365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.809378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.809380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.809385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.809429Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-26T14:16:57.809470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 262214 count 1 nextOffset 56 batches 1
2025-11-26T14:16:57.809478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.809481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.809486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.809623Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 55,1 HeadOffset 55 endOffset 55 curOffset 56 d0000000000_00000000000000000055_00000_0000000001_00000? size 262204 WTime 1764166617809
2025-11-26T14:16:57.809670Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.809683Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 55 partNo 0 count 1 size 262204
2025-11-26T14:16:57.820192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.825259Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 55
    Offset: 55
    WriteTimestampMS: 1764166617809
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 15
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.826534Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.824939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 55 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.824968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.824996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.825007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.825019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 55, partNo: 0, Offset: 55 is stored on disk
2025-11-26T14:16:57.825066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.825070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.825072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.825075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.825077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.825089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.825102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.825160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.827585Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cc8b812a-ffb5238b-40f9283f-8f4dff20_56"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cc8b812a-ffb5238b-40f9283f-8f4dff20_56"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 56
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.830321Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.826893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768841:3555] destroyed
2025-11-26T14:16:57.827358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768846:3558], now have 1 active actors on pipe
2025-11-26T14:16:57.827373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.827379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.827417Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cc8b812a-ffb5238b-40f9283f-8f4dff20_56 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.827430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.827434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.827439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.827441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.827445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.827455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.827457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.827460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.827479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.827493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.830489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768846:3558] destroyed
2025-11-26T14:16:57.830844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768849:3559], now have 1 active actors on pipe
2025-11-26T14:16:57.831283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.831291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.831316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 56 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.831326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.831340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.831343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.831354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.831356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.831360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.831393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-26T14:16:57.831432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 56 PartNo 0 PackedSize 262214 count 1 nextOffset 57 batches 1
2025-11-26T14:16:57.831439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.831441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.831444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.831566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 56,1 HeadOffset 56 endOffset 56 curOffset 57 d0000000000_00000000000000000056_00000_0000000001_00000? size 262204 WTime 1764166617828
2025-11-26T14:16:57.831607Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.831622Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 56 partNo 0 count 1 size 262204
2025-11-26T14:16:57.837486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 56 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.837508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.837533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.837545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.837556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 56, partNo: 0, Offset: 56 is stored on disk
2025-11-26T14:16:57.837590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.837594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.837596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.837598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.837600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.837611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.837621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.837739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.837837Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.838376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768849:3559] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 56
    Offset: 56
    WriteTimestampMS: 1764166617828
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.841254Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.844466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768853:3561], now have 1 active actors on pipe
2025-11-26T14:16:57.856347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.856374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.856427Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|15be0c7b-ea1d0a02-a55e292f-c856fbe0_57 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.856443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.856451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.856458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.856461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.856466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.856480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.856483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.856487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.856508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.856528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.856767Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.857272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768853:3561] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|15be0c7b-ea1d0a02-a55e292f-c856fbe0_57"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|15be0c7b-ea1d0a02-a55e292f-c856fbe0_57"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 57
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.866650Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.869691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768857:3562], now have 1 active actors on pipe
2025-11-26T14:16:57.869843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.869852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.869878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 57 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.869894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.869914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.869920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.869934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.869936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.869943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.869977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-26T14:16:57.870015Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 57 PartNo 0 PackedSize 262214 count 1 nextOffset 58 batches 1
2025-11-26T14:16:57.870022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.870024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.870027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.870152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 57,1 HeadOffset 57 endOffset 57 curOffset 58 d0000000000_00000000000000000057_00000_0000000001_00000? size 262204 WTime 1764166617869
2025-11-26T14:16:57.870197Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.870210Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 57 partNo 0 count 1 size 262204
2025-11-26T14:16:57.873156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 57 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.873179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.873204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.873216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.873226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 57, partNo: 0, Offset: 57 is stored on disk
2025-11-26T14:16:57.873262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.873267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.873269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.873272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.873275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.873284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.873295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.873355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 57
    Offset: 57
    WriteTimestampMS: 1764166617869
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.875414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768857:3562] destroyed
2025-11-26T14:16:57.875427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768861:3565], now have 1 active actors on pipe
2025-11-26T14:16:57.875436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.875442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.875471Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e393f29-3dfcb501-d5eae4b8-f7d0f688_58 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.875481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.875485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.875489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.875491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.875494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.875506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.875509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.875511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.875526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.875543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e393f29-3dfcb501-d5eae4b8-f7d0f688_58"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:57.873532Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.874759Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.875692Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e393f29-3dfcb501-d5eae4b8-f7d0f688_58"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 58
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.878427Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.883041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768861:3565] destroyed
2025-11-26T14:16:57.883060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768864:3566], now have 1 active actors on pipe
2025-11-26T14:16:57.883191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.883203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.883228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 58 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.883245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.883265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.883269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.883282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.883284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.883290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.883325Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-26T14:16:57.883366Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 58 PartNo 0 PackedSize 262214 count 1 nextOffset 59 batches 1
2025-11-26T14:16:57.883375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.883378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.883381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.883512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 58,1 HeadOffset 58 endOffset 58 curOffset 59 d0000000000_00000000000000000058_00000_0000000001_00000? size 262204 WTime 1764166617880
2025-11-26T14:16:57.883565Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.883577Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 58 partNo 0 count 1 size 262204
2025-11-26T14:16:57.886017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 58 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.886041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.886068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.886080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.886091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 58, partNo: 0, Offset: 58 is stored on disk
2025-11-26T14:16:57.886140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.886144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.886146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.886149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.886151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.886162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.886177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.886315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 58
    Offset: 58
    WriteTimestampMS: 1764166617880
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.886434Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.887812Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.888312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768864:3566] destroyed
2025-11-26T14:16:57.888329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768868:3568], now have 1 active actors on pipe
2025-11-26T14:16:57.888460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.888469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.888503Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fb73916-b478559e-19d2ef91-6eafad62_59 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.888513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.888516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.888524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.888526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.888531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.888542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.888545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.888547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.888561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.888573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.888695Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.888960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768868:3568] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fb73916-b478559e-19d2ef91-6eafad62_59"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fb73916-b478559e-19d2ef91-6eafad62_59"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 59
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.896343Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.904536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768871:3569], now have 1 active actors on pipe
2025-11-26T14:16:57.912905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.912927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.912961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 59 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.912979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.912998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.913002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.913013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.913015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.913020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.913054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-26T14:16:57.913093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 59 PartNo 0 PackedSize 262214 count 1 nextOffset 60 batches 1
2025-11-26T14:16:57.913100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.913103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.913106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.913244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 59,1 HeadOffset 59 endOffset 59 curOffset 60 d0000000000_00000000000000000059_00000_0000000001_00000? size 262204 WTime 1764166617912
2025-11-26T14:16:57.913289Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.913302Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 59 partNo 0 count 1 size 262204
2025-11-26T14:16:57.924556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:57.933089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 59 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.933116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.933146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.933159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.933173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 59, partNo: 0, Offset: 59 is stored on disk
2025-11-26T14:16:57.933222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.933226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.933229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.933232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.933234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.933244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.933257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.933461Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:57.933391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.933868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768871:3569] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 59
    Offset: 59
    WriteTimestampMS: 1764166617912
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 19
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.937638Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.938372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768879:3572], now have 1 active actors on pipe
2025-11-26T14:16:57.944413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.944435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.944479Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|808dd9cc-95d745f6-e8d771b3-63232470_60 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.944492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.944497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.944504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.944506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.944511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.944523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.944526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.944530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.944548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.944569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.947483Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|808dd9cc-95d745f6-e8d771b3-63232470_60"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:57.948438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768879:3572] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|808dd9cc-95d745f6-e8d771b3-63232470_60"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 60
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.953360Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.956821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768888:3577], now have 1 active actors on pipe
2025-11-26T14:16:57.964671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.964691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.964721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 60 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.964741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.964759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.964764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.964780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.964783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.964791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.964827Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-26T14:16:57.964866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 60 PartNo 0 PackedSize 262214 count 1 nextOffset 61 batches 1
2025-11-26T14:16:57.964875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.964878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.964881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.965007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 60,1 HeadOffset 60 endOffset 60 curOffset 61 d0000000000_00000000000000000060_00000_0000000001_00000? size 262204 WTime 1764166617963
2025-11-26T14:16:57.965042Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.965055Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 60 partNo 0 count 1 size 262204
2025-11-26T14:16:57.977438Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 60
    Offset: 60
    WriteTimestampMS: 1764166617963
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.977114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 60 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.977152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.977179Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.977192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.977205Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 60, partNo: 0, Offset: 60 is stored on disk
2025-11-26T14:16:57.977259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.977263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.977266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.977271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.977274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.977287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.977300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.977363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.978996Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.979573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768888:3577] destroyed
2025-11-26T14:16:57.979596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768892:3579], now have 1 active actors on pipe
2025-11-26T14:16:57.979608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.979614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.979646Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c30c19f0-754e3fa3-e4fff4ad-a94a6098_61 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.979654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.979658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.979663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.979665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.979670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.979678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.979683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.979687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.979712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.979727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.979905Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c30c19f0-754e3fa3-e4fff4ad-a94a6098_61"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c30c19f0-754e3fa3-e4fff4ad-a94a6098_61"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 61
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.982769Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.982912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768892:3579] destroyed
2025-11-26T14:16:57.983961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768895:3580], now have 1 active actors on pipe
2025-11-26T14:16:57.984078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.984087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.984112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 61 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:57.984124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:57.984139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.984144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.984155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.984159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.984163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.984193Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-26T14:16:57.984237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 61 PartNo 0 PackedSize 262214 count 1 nextOffset 62 batches 1
2025-11-26T14:16:57.984243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.984246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.984250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.984417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 61,1 HeadOffset 61 endOffset 61 curOffset 62 d0000000000_00000000000000000061_00000_0000000001_00000? size 262204 WTime 1764166617984
2025-11-26T14:16:57.984465Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:57.984478Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 61 partNo 0 count 1 size 262204
2025-11-26T14:16:57.987059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 61 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:57.987081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:57.987160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:57.992374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:57.992425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:57.992440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:57.992458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 61, partNo: 0, Offset: 61 is stored on disk
2025-11-26T14:16:57.992516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.992520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.992523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.992527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:57.992530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.992543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:57.992564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.993060Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 61
    Offset: 61
    WriteTimestampMS: 1764166617978
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:57.994352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768895:3580] destroyed
2025-11-26T14:16:57.994435Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.995046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768899:3582], now have 1 active actors on pipe
2025-11-26T14:16:57.995220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.995230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:57.995263Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6a12dfff-49b9a4f3-c820a10-d4bf57c8_62 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:57.995275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:57.995280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:57.995285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:57.995288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:57.995293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:57.995304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:57.995307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:57.995310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:57.995326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:57.995340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:57.995468Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6a12dfff-49b9a4f3-c820a10-d4bf57c8_62"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:57.996676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768899:3582] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6a12dfff-49b9a4f3-c820a10-d4bf57c8_62"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 62
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:57.998939Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:57.999862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037926988768902:3583], now have 1 active actors on pipe
2025-11-26T14:16:57.999982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:57.999990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.000017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 62 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.000028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.000046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.000057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.000067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.000069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.000074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.000102Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-26T14:16:58.000146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 62 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-26T14:16:58.000152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.000155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.000157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.000313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,1 HeadOffset 62 endOffset 62 curOffset 63 d0000000000_00000000000000000062_00000_0000000001_00000? size 262204 WTime 1764166617992
2025-11-26T14:16:58.000362Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.000375Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 1 size 262204
2025-11-26T14:16:58.003068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.003093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.003116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.003127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.003138Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 62, partNo: 0, Offset: 62 is stored on disk
2025-11-26T14:16:58.003179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.003182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.003187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.003190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.003192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.003201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.003212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.003273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.003417Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 62
    Offset: 62
    WriteTimestampMS: 1764166617992
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.004787Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.005463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037926988768902:3583] destroyed
2025-11-26T14:16:58.005478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736202:3585], now have 1 active actors on pipe
2025-11-26T14:16:58.005488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.005494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.005524Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|aa2530ed-d6c37cc5-9e3a28d4-973a0354_63 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.005532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.005536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.005540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.005542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.005546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.005554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.005557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.005560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.005571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.005581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.005726Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|aa2530ed-d6c37cc5-9e3a28d4-973a0354_63"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|aa2530ed-d6c37cc5-9e3a28d4-973a0354_63"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 63
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.008600Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.009697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736202:3585] destroyed
2025-11-26T14:16:58.009713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736205:3586], now have 1 active actors on pipe
2025-11-26T14:16:58.009823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.009831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.009856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 63 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.009871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.009884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.009888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.009896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.009899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.009904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.009929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-26T14:16:58.009964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 262214 count 1 nextOffset 64 batches 1
2025-11-26T14:16:58.009970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.009974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.009977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.010101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 63,1 HeadOffset 63 endOffset 63 curOffset 64 d0000000000_00000000000000000063_00000_0000000001_00000? size 262204 WTime 1764166618008
2025-11-26T14:16:58.010145Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.010159Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 63 partNo 0 count 1 size 262204
2025-11-26T14:16:58.012577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 63 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.012603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.012625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.012636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.012646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 63, partNo: 0, Offset: 63 is stored on disk
2025-11-26T14:16:58.012682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.012686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.012688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.012691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.012694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.012705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000032_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000033_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000034_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000035_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000036_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000037_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000038_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000039_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000040_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000041_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000042_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000043_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000044_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000045_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000046_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000047_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000048_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000049_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000050_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000051_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000052_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000053_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000054_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000055_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000056_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000057_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000058_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000059_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000060_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000061_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000062_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000063_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.012811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-26T14:16:58.012824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.012839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-26T14:16:58.012843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-26T14:16:58.012853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-26T14:16:58.012863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-26T14:16:58.012878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-26T14:16:58.012886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-26T14:16:58.012903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-26T14:16:58.012912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-26T14:16:58.012920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-26T14:16:58.012928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-26T14:16:58.012954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-26T14:16:58.012964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-26T14:16:58.012977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-26T14:16:58.012986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-26T14:16:58.013253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.013390Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 63
    Offset: 63
    WriteTimestampMS: 1764166618008
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.014769Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.015423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-26T14:16:58.015442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-26T14:16:58.015453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-26T14:16:58.015479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-26T14:16:58.015526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-26T14:16:58.015539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-26T14:16:58.015549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-26T14:16:58.015559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-26T14:16:58.015569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-26T14:16:58.015579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-26T14:16:58.015590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-26T14:16:58.015610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-26T14:16:58.015620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-26T14:16:58.015630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-26T14:16:58.015639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-26T14:16:58.015649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-26T14:16:58.015659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-26T14:16:58.015669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-26T14:16:58.015678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-26T14:16:58.015689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-26T14:16:58.015705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736205:3586] destroyed
2025-11-26T14:16:58.015712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736209:3588], now have 1 active actors on pipe
2025-11-26T14:16:58.015730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.015736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.015756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 32 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 33 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 34 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015767Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 35 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 36 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 37 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 38 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 39 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 40 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015791Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 41 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 42 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015802Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 43 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 44 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 45 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 46 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 47 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 48 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 49 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 50 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 51 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 52 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 53 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 54 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 55 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 56 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 57 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 58 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 59 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 60 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 61 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.015873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 62 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.015876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 63 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.015893Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-26T14:16:58.015948Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2ba64d50-81e64ac3-d3d7d55b-58d818b8_64 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.015956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.015959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.015963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.015966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.015970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.015980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.015983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.015985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.015997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.016006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-26T14:16:58.016014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-26T14:16:58.016016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.016020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000032_00000_0000000001_00000?, parameters.CurOffset=32
2025-11-26T14:16:58.016148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 32.0/1
2025-11-26T14:16:58.016156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-26T14:16:58.016189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-26T14:16:58.016193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 32:0 appended
2025-11-26T14:16:58.016201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-26T14:16:58.016203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.016205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000033_00000_0000000001_00000?, parameters.CurOffset=33
2025-11-26T14:16:58.016337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 33.0/1
2025-11-26T14:16:58.016343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-26T14:16:58.016598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 524387 count 2 nextOffset 34 batches 1
2025-11-26T14:16:58.016602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 33:0 appended
2025-11-26T14:16:58.016983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.016991Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.016996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017005Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017009Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017018Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017030Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017044Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017048Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017052Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017076Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017079Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017088Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017094Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017100Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.017311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-26T14:16:58.017317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.017321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000034_00000_0000000001_00000?, parameters.CurOffset=34
2025-11-26T14:16:58.017421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 34.0/1
2025-11-26T14:16:58.017428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-26T14:16:58.017445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 786601 count 3 nextOffset 35 batches 2
2025-11-26T14:16:58.017447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 34:0 appended
2025-11-26T14:16:58.017452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-26T14:16:58.017454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.017457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000035_00000_0000000001_00000?, parameters.CurOffset=35
2025-11-26T14:16:58.017531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 35.0/1
2025-11-26T14:16:58.017535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-26T14:16:58.018097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1048774 count 4 nextOffset 36 batches 2
2025-11-26T14:16:58.018101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 35:0 appended
2025-11-26T14:16:58.018106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-26T14:16:58.018109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.018112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000036_00000_0000000001_00000?, parameters.CurOffset=36
2025-11-26T14:16:58.018205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 36.0/1
2025-11-26T14:16:58.018209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-26T14:16:58.018248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1310988 count 5 nextOffset 37 batches 3
2025-11-26T14:16:58.018250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 36:0 appended
2025-11-26T14:16:58.018256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-26T14:16:58.018258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.018261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000037_00000_0000000001_00000?, parameters.CurOffset=37
2025-11-26T14:16:58.018329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 37.0/1
2025-11-26T14:16:58.018333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-26T14:16:58.018861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1573161 count 6 nextOffset 38 batches 3
2025-11-26T14:16:58.018864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 37:0 appended
2025-11-26T14:16:58.018868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-26T14:16:58.018870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.018873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000038_00000_0000000001_00000?, parameters.CurOffset=38
2025-11-26T14:16:58.018953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 38.0/1
2025-11-26T14:16:58.018956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-26T14:16:58.018968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1835375 count 7 nextOffset 39 batches 4
2025-11-26T14:16:58.018971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 38:0 appended
2025-11-26T14:16:58.018975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-26T14:16:58.018977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.018980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000039_00000_0000000001_00000?, parameters.CurOffset=39
2025-11-26T14:16:58.019049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 39.0/1
2025-11-26T14:16:58.019052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-26T14:16:58.019576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2097548 count 8 nextOffset 40 batches 4
2025-11-26T14:16:58.019579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 39:0 appended
2025-11-26T14:16:58.019585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-26T14:16:58.019587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.019590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000040_00000_0000000001_00000?, parameters.CurOffset=40
2025-11-26T14:16:58.019668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 40.0/1
2025-11-26T14:16:58.019672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-26T14:16:58.019683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2359762 count 9 nextOffset 41 batches 5
2025-11-26T14:16:58.019686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 40:0 appended
2025-11-26T14:16:58.019690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-26T14:16:58.019692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.019696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000041_00000_0000000001_00000?, parameters.CurOffset=41
2025-11-26T14:16:58.019763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 41.0/1
2025-11-26T14:16:58.019766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-26T14:16:58.020316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2621935 count 10 nextOffset 42 batches 5
2025-11-26T14:16:58.020319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 41:0 appended
2025-11-26T14:16:58.020325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-26T14:16:58.020327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.020330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000042_00000_0000000001_00000?, parameters.CurOffset=42
2025-11-26T14:16:58.020410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 42.0/1
2025-11-26T14:16:58.020413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-26T14:16:58.020424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2884149 count 11 nextOffset 43 batches 6
2025-11-26T14:16:58.020427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 42:0 appended
2025-11-26T14:16:58.020430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-26T14:16:58.020434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.020437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000043_00000_0000000001_00000?, parameters.CurOffset=43
2025-11-26T14:16:58.020507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 43.0/1
2025-11-26T14:16:58.020510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-26T14:16:58.020645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3146322 count 12 nextOffset 44 batches 6
2025-11-26T14:16:58.020648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 43:0 appended
2025-11-26T14:16:58.020653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-26T14:16:58.020655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.020658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000044_00000_0000000001_00000?, parameters.CurOffset=44
2025-11-26T14:16:58.020740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 44.0/1
2025-11-26T14:16:58.020744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-26T14:16:58.020753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3408536 count 13 nextOffset 45 batches 7
2025-11-26T14:16:58.020755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 44:0 appended
2025-11-26T14:16:58.020758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-26T14:16:58.020760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.020763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000045_00000_0000000001_00000?, parameters.CurOffset=45
2025-11-26T14:16:58.020833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 45.0/1
2025-11-26T14:16:58.020840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-26T14:16:58.020965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3670709 count 14 nextOffset 46 batches 7
2025-11-26T14:16:58.020967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 45:0 appended
2025-11-26T14:16:58.020972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-26T14:16:58.020974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.020980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000046_00000_0000000001_00000?, parameters.CurOffset=46
2025-11-26T14:16:58.021053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 46.0/1
2025-11-26T14:16:58.021056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-26T14:16:58.021067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3932923 count 15 nextOffset 47 batches 8
2025-11-26T14:16:58.021070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 46:0 appended
2025-11-26T14:16:58.021073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-26T14:16:58.021074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.021077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000047_00000_0000000001_00000?, parameters.CurOffset=47
2025-11-26T14:16:58.021144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 47.0/1
2025-11-26T14:16:58.021147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-26T14:16:58.021265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4195096 count 16 nextOffset 48 batches 8
2025-11-26T14:16:58.021268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 47:0 appended
2025-11-26T14:16:58.021271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-26T14:16:58.021274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.021276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000048_00000_0000000001_00000?, parameters.CurOffset=48
2025-11-26T14:16:58.021347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 48.0/1
2025-11-26T14:16:58.021351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-26T14:16:58.021360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4457310 count 17 nextOffset 49 batches 9
2025-11-26T14:16:58.021362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 48:0 appended
2025-11-26T14:16:58.021365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-26T14:16:58.021367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.021369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000049_00000_0000000001_00000?, parameters.CurOffset=49
2025-11-26T14:16:58.021438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 49.0/1
2025-11-26T14:16:58.021441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-26T14:16:58.021563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4719483 count 18 nextOffset 50 batches 9
2025-11-26T14:16:58.021565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 49:0 appended
2025-11-26T14:16:58.021570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-26T14:16:58.021573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.021576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000050_00000_0000000001_00000?, parameters.CurOffset=50
2025-11-26T14:16:58.021644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 50.0/1
2025-11-26T14:16:58.021649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-26T14:16:58.021658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4981697 count 19 nextOffset 51 batches 10
2025-11-26T14:16:58.021660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 50:0 appended
2025-11-26T14:16:58.021663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-26T14:16:58.021665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.021668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000051_00000_0000000001_00000?, parameters.CurOffset=51
2025-11-26T14:16:58.021734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 51.0/1
2025-11-26T14:16:58.021737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-26T14:16:58.022870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5243870 count 20 nextOffset 52 batches 10
2025-11-26T14:16:58.022879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 51:0 appended
2025-11-26T14:16:58.022885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-26T14:16:58.022887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.022891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000052_00000_0000000001_00000?, parameters.CurOffset=52
2025-11-26T14:16:58.022970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 52.0/1
2025-11-26T14:16:58.022974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-26T14:16:58.022998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5506084 count 21 nextOffset 53 batches 11
2025-11-26T14:16:58.023001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 52:0 appended
2025-11-26T14:16:58.023004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-26T14:16:58.023006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.023008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000053_00000_0000000001_00000?, parameters.CurOffset=53
2025-11-26T14:16:58.023076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 53.0/1
2025-11-26T14:16:58.023080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-26T14:16:58.023594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5768257 count 22 nextOffset 54 batches 11
2025-11-26T14:16:58.023597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 53:0 appended
2025-11-26T14:16:58.023602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-26T14:16:58.023603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.023605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000054_00000_0000000001_00000?, parameters.CurOffset=54
2025-11-26T14:16:58.023675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 54.0/1
2025-11-26T14:16:58.023678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-26T14:16:58.023687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6030471 count 23 nextOffset 55 batches 12
2025-11-26T14:16:58.023689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 54:0 appended
2025-11-26T14:16:58.023694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-26T14:16:58.023695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.023698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000055_00000_0000000001_00000?, parameters.CurOffset=55
2025-11-26T14:16:58.023761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 55.0/1
2025-11-26T14:16:58.023764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-26T14:16:58.024327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6292644 count 24 nextOffset 56 batches 12
2025-11-26T14:16:58.024333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 55:0 appended
2025-11-26T14:16:58.024339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-26T14:16:58.024341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.024344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000056_00000_0000000001_00000?, parameters.CurOffset=56
2025-11-26T14:16:58.024420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 56.0/1
2025-11-26T14:16:58.024424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-26T14:16:58.024438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6554858 count 25 nextOffset 57 batches 13
2025-11-26T14:16:58.024440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 56:0 appended
2025-11-26T14:16:58.024444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-26T14:16:58.024446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.024449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000057_00000_0000000001_00000?, parameters.CurOffset=57
2025-11-26T14:16:58.024518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 57.0/1
2025-11-26T14:16:58.024521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-26T14:16:58.025047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6817031 count 26 nextOffset 58 batches 13
2025-11-26T14:16:58.025052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 57:0 appended
2025-11-26T14:16:58.025057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-26T14:16:58.025060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.025063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000058_00000_0000000001_00000?, parameters.CurOffset=58
2025-11-26T14:16:58.025139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 58.0/1
2025-11-26T14:16:58.025143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-26T14:16:58.025235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7079245 count 27 nextOffset 59 batches 14
2025-11-26T14:16:58.025237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 58:0 appended
2025-11-26T14:16:58.025242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-26T14:16:58.025245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.025247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000059_00000_0000000001_00000?, parameters.CurOffset=59
2025-11-26T14:16:58.025318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 59.0/1
2025-11-26T14:16:58.025321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-26T14:16:58.025826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7341418 count 28 nextOffset 60 batches 14
2025-11-26T14:16:58.025830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 59:0 appended
2025-11-26T14:16:58.025835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-26T14:16:58.025838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.025841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000060_00000_0000000001_00000?, parameters.CurOffset=60
2025-11-26T14:16:58.026361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 60.0/1
2025-11-26T14:16:58.026374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-26T14:16:58.026390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7603632 count 29 nextOffset 61 batches 15
2025-11-26T14:16:58.026393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 60:0 appended
2025-11-26T14:16:58.026397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-26T14:16:58.026399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.026403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000061_00000_0000000001_00000?, parameters.CurOffset=61
2025-11-26T14:16:58.026479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 61.0/1
2025-11-26T14:16:58.026483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-26T14:16:58.027015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7865805 count 30 nextOffset 62 batches 15
2025-11-26T14:16:58.027020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 61:0 appended
2025-11-26T14:16:58.027025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-26T14:16:58.027027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.027038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000062_00000_0000000001_00000?, parameters.CurOffset=62
2025-11-26T14:16:58.027097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 62.0/1
2025-11-26T14:16:58.027102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-26T14:16:58.034320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 62 partNo 0 result is x0000000000_00000000000000000031_00000_0000000031_00000 size 8128009
2025-11-26T14:16:58.034345Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000031_00000_0000000031_00000 new key d0000000000_00000000000000000031_00000_0000000031_00000 size 8128009 WTime 1764166618032
2025-11-26T14:16:58.034382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 62 partNo 0 FormedBlobsCount 1 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-26T14:16:58.034389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 62:0 appended
2025-11-26T14:16:58.034398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-26T14:16:58.034401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.034405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000063_00000_0000000001_00000?, parameters.CurOffset=63
2025-11-26T14:16:58.034689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 63.0/1
2025-11-26T14:16:58.034697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-26T14:16:58.035471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 524387 count 2 nextOffset 64 batches 1
2025-11-26T14:16:58.035476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 63:0 appended
2025-11-26T14:16:58.035537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,2 HeadOffset 31 endOffset 32 curOffset 64 d0000000000_00000000000000000062_00000_0000000002_00000| size 524387 WTime 1764166618035
2025-11-26T14:16:58.036014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.036051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.036054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.036056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.036060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.036064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.036079Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.036092Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 31 size 8128009
2025-11-26T14:16:58.036101Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 2 size 524387
2025-11-26T14:16:58.036644Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2ba64d50-81e64ac3-d3d7d55b-58d818b8_64"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2ba64d50-81e64ac3-d3d7d55b-58d818b8_64"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 64
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.042184Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.044880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736209:3588] destroyed
2025-11-26T14:16:58.044903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736214:3591], now have 1 active actors on pipe
2025-11-26T14:16:58.045036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.045043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.045068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 64 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.045084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.045097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.045101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.045111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.045113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.045119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.045151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-26T14:16:58.045187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-26T14:16:58.045198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.045201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.045204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.045526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 64,1 HeadOffset 64 endOffset 64 curOffset 65 d0000000000_00000000000000000064_00000_0000000001_00000? size 262204 WTime 1764166618045
2025-11-26T14:16:58.045569Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.045582Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 64 partNo 0 count 1 size 262204
2025-11-26T14:16:58.068480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 64 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.068520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.068552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.068565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.068578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 64, partNo: 0, Offset: 64 is stored on disk
2025-11-26T14:16:58.068626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.068630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.068632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.068635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.068638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.068649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:178: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-11-26T14:16:58.068664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.068994Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 64
    Offset: 64
    WriteTimestampMS: 1764166618044
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 23
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.068834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.069718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736214:3591] destroyed
2025-11-26T14:16:58.070260Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.080340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736224:3594], now have 1 active actors on pipe
2025-11-26T14:16:58.080390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.080402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.080541Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5b9cd09e-61ae39b2-98da0d87-276fb7aa_65 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.080557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.080562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.080567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.080570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.080575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.080588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.080590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.080597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.080617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.080638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 31 size 8128009 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.080645Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 2 size 524387 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.080674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.080684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.081098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.081100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.081102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.081105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.081107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.081155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.081163Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.081171Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000032_00000_0000000001_00000?(+) to d0000000000_00000000000000000032_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081173Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000033_00000_0000000001_00000?(+) to d0000000000_00000000000000000033_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081175Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000034_00000_0000000001_00000?(+) to d0000000000_00000000000000000034_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081178Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000035_00000_0000000001_00000?(+) to d0000000000_00000000000000000035_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081180Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000036_00000_0000000001_00000?(+) to d0000000000_00000000000000000036_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081181Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000037_00000_0000000001_00000?(+) to d0000000000_00000000000000000037_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081183Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000038_00000_0000000001_00000?(+) to d0000000000_00000000000000000038_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081185Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000039_00000_0000000001_00000?(+) to d0000000000_00000000000000000039_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081187Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000040_00000_0000000001_00000?(+) to d0000000000_00000000000000000040_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081189Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000041_00000_0000000001_00000?(+) to d0000000000_00000000000000000041_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081191Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000042_00000_0000000001_00000?(+) to d0000000000_00000000000000000042_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081193Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000043_00000_0000000001_00000?(+) to d0000000000_00000000000000000043_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081195Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000044_00000_0000000001_00000?(+) to d0000000000_00000000000000000044_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081197Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000045_00000_0000000001_00000?(+) to d0000000000_00000000000000000045_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081199Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000046_00000_0000000001_00000?(+) to d0000000000_00000000000000000046_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081201Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000047_00000_0000000001_00000?(+) to d0000000000_00000000000000000047_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081206Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000048_00000_0000000001_00000?(+) to d0000000000_00000000000000000048_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081207Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000049_00000_0000000001_00000?(+) to d0000000000_00000000000000000049_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081209Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000050_00000_0000000001_00000?(+) to d0000000000_00000000000000000050_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081211Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000051_00000_0000000001_00000?(+) to d0000000000_00000000000000000051_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081213Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000052_00000_0000000001_00000?(+) to d0000000000_00000000000000000052_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081215Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000053_00000_0000000001_00000?(+) to d0000000000_00000000000000000053_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081218Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000054_00000_0000000001_00000?(+) to d0000000000_00000000000000000054_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081221Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000055_00000_0000000001_00000?(+) to d0000000000_00000000000000000055_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081223Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000056_00000_0000000001_00000?(+) to d0000000000_00000000000000000056_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081225Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000057_00000_0000000001_00000?(+) to d0000000000_00000000000000000057_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081226Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000058_00000_0000000001_00000?(+) to d0000000000_00000000000000000058_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081231Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000059_00000_0000000001_00000?(+) to d0000000000_00000000000000000059_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081233Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000060_00000_0000000001_00000?(+) to d0000000000_00000000000000000060_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081235Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000061_00000_0000000001_00000?(+) to d0000000000_00000000000000000061_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081237Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000001_00000?(+) to d0000000000_00000000000000000062_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081238Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000063_00000_0000000001_00000?(+) to d0000000000_00000000000000000063_00000_0000000001_00000?(+)
2025-11-26T14:16:58.081240Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000|(+) to d0000000000_00000000000000000031_00000_0000000001_00000|(+)
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5b9cd09e-61ae39b2-98da0d87-276fb7aa_65"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5b9cd09e-61ae39b2-98da0d87-276fb7aa_65"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 65
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.084243Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-26T14:16:58.084279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-26T14:16:58.088247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736224:3594] destroyed
2025-11-26T14:16:58.084701Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.088376Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.089087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 32 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 33 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 34 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 35 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 36 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089129Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 37 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 38 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 39 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 40 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 41 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 42 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 43 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 44 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 45 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 46 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 47 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 48 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 49 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 50 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 51 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089212Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 52 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 53 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 54 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089229Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 55 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 56 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 57 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089246Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 58 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 59 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089256Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 60 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 61 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 62 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 63 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.089301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.089321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.089332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.089335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.089337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.089340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.089342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.089352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.089412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089432Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089436Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089442Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089484Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089496Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.089548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-26T14:16:58.090248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736228:3597], now have 1 active actors on pipe
2025-11-26T14:16:58.091373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.091385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.091411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 65 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.091425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.091439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.091447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.091458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.091460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.091466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.091494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-26T14:16:58.091530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 262214 count 1 nextOffset 66 batches 1
2025-11-26T14:16:58.091538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.091540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.091544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.091674Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 65,1 HeadOffset 65 endOffset 65 curOffset 66 d0000000000_00000000000000000065_00000_0000000001_00000? size 262204 WTime 1764166618091
2025-11-26T14:16:58.091710Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.091724Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 65 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 65
    Offset: 65
    WriteTimestampMS: 1764166618091
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 12
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.104836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 65 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.104876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.104917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.104933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.104948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 65, partNo: 0, Offset: 65 is stored on disk
2025-11-26T14:16:58.104997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.105002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.105004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.105008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.105011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.105022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.105036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.105178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.105285Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.106543Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.107232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736228:3597] destroyed
2025-11-26T14:16:58.107254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736233:3600], now have 1 active actors on pipe
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a3650df4-50f107f-7bc8c61e-5f22ee92_66"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a3650df4-50f107f-7bc8c61e-5f22ee92_66"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 66
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.107556Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.107358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.107371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.107403Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a3650df4-50f107f-7bc8c61e-5f22ee92_66 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.107413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.107417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.107425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.107430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.107434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.107444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.107447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.107450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.107463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.107477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.110525Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.111011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736233:3600] destroyed
2025-11-26T14:16:58.111030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736236:3601], now have 1 active actors on pipe
2025-11-26T14:16:58.111524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.111536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.111558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 66 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.111568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.111582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.111586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.111595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.111597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.111604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.111631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-26T14:16:58.111662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 66 PartNo 0 PackedSize 262214 count 1 nextOffset 67 batches 1
2025-11-26T14:16:58.111669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.111672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.111675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.111796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 66,1 HeadOffset 66 endOffset 66 curOffset 67 d0000000000_00000000000000000066_00000_0000000001_00000? size 262204 WTime 1764166618111
2025-11-26T14:16:58.111834Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.111847Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 66 partNo 0 count 1 size 262204
2025-11-26T14:16:58.124998Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 66 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.125027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.125052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.125063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.125081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 66, partNo: 0, Offset: 66 is stored on disk
2025-11-26T14:16:58.125135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.125138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.125140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.125142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.125145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.125157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.125170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.125449Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 66
    Offset: 66
    WriteTimestampMS: 1764166618111
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.125356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.126852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736236:3601] destroyed
2025-11-26T14:16:58.126923Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.127491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736241:3604], now have 1 active actors on pipe
2025-11-26T14:16:58.127673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.127684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.127716Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b7ebeb9e-363dec7a-5ae8d236-d269e17e_67 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.127727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.127730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.127736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.127738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.127743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.127753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.127756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.127760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.127777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.127793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b7ebeb9e-363dec7a-5ae8d236-d269e17e_67"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:58.127922Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b7ebeb9e-363dec7a-5ae8d236-d269e17e_67"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 67
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.130724Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.130900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736241:3604] destroyed
2025-11-26T14:16:58.131221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736244:3605], now have 1 active actors on pipe
2025-11-26T14:16:58.131666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.131678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.131701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 67 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.131712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.131725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.131728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.131737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.131739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.131744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.131772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-26T14:16:58.131804Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 67 PartNo 0 PackedSize 262214 count 1 nextOffset 68 batches 1
2025-11-26T14:16:58.131810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.131812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.131815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.131925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 67,1 HeadOffset 67 endOffset 67 curOffset 68 d0000000000_00000000000000000067_00000_0000000001_00000? size 262204 WTime 1764166618131
2025-11-26T14:16:58.131960Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.131973Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 67 partNo 0 count 1 size 262204
2025-11-26T14:16:58.138084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:58.144875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 67 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.144906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.144934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.144946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.144962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 67, partNo: 0, Offset: 67 is stored on disk
2025-11-26T14:16:58.145015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.145019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.145021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.145024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.145026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.145039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.145052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.145199Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.145314Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 67
    Offset: 67
    WriteTimestampMS: 1764166618131
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.146681Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.147112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736244:3605] destroyed
2025-11-26T14:16:58.147598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736248:3607], now have 1 active actors on pipe
2025-11-26T14:16:58.147615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.147622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.147656Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e5c2228b-37b3375d-dc2b4f9f-372b1a90_68 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.147665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.147672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.147677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.147680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.147684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.147697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.147701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.147704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.147718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.147730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.147879Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e5c2228b-37b3375d-dc2b4f9f-372b1a90_68"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e5c2228b-37b3375d-dc2b4f9f-372b1a90_68"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 68
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.150792Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.150971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736248:3607] destroyed
2025-11-26T14:16:58.151360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736251:3608], now have 1 active actors on pipe
2025-11-26T14:16:58.151902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.151917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.151938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 68 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.151949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.151963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.151966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.151977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.151979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.151983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.152008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-26T14:16:58.152040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 68 PartNo 0 PackedSize 262214 count 1 nextOffset 69 batches 1
2025-11-26T14:16:58.152046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.152049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.152051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.152170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 68,1 HeadOffset 68 endOffset 68 curOffset 69 d0000000000_00000000000000000068_00000_0000000001_00000? size 262204 WTime 1764166618152
2025-11-26T14:16:58.152210Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.152224Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 68 partNo 0 count 1 size 262204
2025-11-26T14:16:58.164873Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 68 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.164907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.164934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.164946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.164959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 68, partNo: 0, Offset: 68 is stored on disk
2025-11-26T14:16:58.165010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.165018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.165020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.165027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.165029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.165039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.165054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.165204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.165335Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 68
    Offset: 68
    WriteTimestampMS: 1764166618151
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 12
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.166695Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.167072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736251:3608] destroyed
2025-11-26T14:16:58.167564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736255:3610], now have 1 active actors on pipe
2025-11-26T14:16:58.167581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.167588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.167620Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ff5f184e-d8f33412-b1da5dd1-879b00c7_69 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.167631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.167635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.167639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.167642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.167646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.167656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.167660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.167662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.167677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.167693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.167839Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ff5f184e-d8f33412-b1da5dd1-879b00c7_69"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ff5f184e-d8f33412-b1da5dd1-879b00c7_69"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 69
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.170978Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.171155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736255:3610] destroyed
2025-11-26T14:16:58.171534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736258:3611], now have 1 active actors on pipe
2025-11-26T14:16:58.172029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.172043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.172066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 69 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.172076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.172091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.172095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.172109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.172112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.172117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.172147Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-26T14:16:58.172181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 69 PartNo 0 PackedSize 262214 count 1 nextOffset 70 batches 1
2025-11-26T14:16:58.172189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.172192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.172197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.172342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 69,1 HeadOffset 69 endOffset 69 curOffset 70 d0000000000_00000000000000000069_00000_0000000001_00000? size 262204 WTime 1764166618171
2025-11-26T14:16:58.172389Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.172403Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 69 partNo 0 count 1 size 262204
2025-11-26T14:16:58.188744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 69 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.188780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.188807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.188820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.188834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 69, partNo: 0, Offset: 69 is stored on disk
2025-11-26T14:16:58.188890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.188895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.188898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.188901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.188904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.188915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.188928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.189185Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 69
    Offset: 69
    WriteTimestampMS: 1764166618171
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 14
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.189079Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.190619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736258:3611] destroyed
2025-11-26T14:16:58.190683Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.191237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736262:3613], now have 1 active actors on pipe
2025-11-26T14:16:58.191422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.191433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.191465Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|91715cc9-f593da79-9e192618-29904e07_70 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.191476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.191479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.191483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.191485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.191490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.191499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.191502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.191504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.191520Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.191534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.191673Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|91715cc9-f593da79-9e192618-29904e07_70"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|91715cc9-f593da79-9e192618-29904e07_70"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 70
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.194278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736262:3613] destroyed
2025-11-26T14:16:58.194477Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.195263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736265:3614], now have 1 active actors on pipe
2025-11-26T14:16:58.195344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.195350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.195369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 70 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.195376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.195389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.195395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.195403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.195405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.195409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.195435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-26T14:16:58.195474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 70 PartNo 0 PackedSize 262214 count 1 nextOffset 71 batches 1
2025-11-26T14:16:58.195480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.195483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.195485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.195588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 70,1 HeadOffset 70 endOffset 70 curOffset 71 d0000000000_00000000000000000070_00000_0000000001_00000? size 262204 WTime 1764166618192
2025-11-26T14:16:58.195623Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.195634Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 70 partNo 0 count 1 size 262204
2025-11-26T14:16:58.197994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 70 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.198013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.198034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.198043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.198052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 70, partNo: 0, Offset: 70 is stored on disk
2025-11-26T14:16:58.198144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.198210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.198215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.198217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.198220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.198221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.198231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.198241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.198364Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 70
    Offset: 70
    WriteTimestampMS: 1764166618192
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.199484Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.200087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736265:3614] destroyed
2025-11-26T14:16:58.200099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736269:3616], now have 1 active actors on pipe
2025-11-26T14:16:58.200123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.200130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.200154Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|262bd5d5-49e112d2-a51713ba-1f228b99_71 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.200161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.200164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.200167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.200169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.200172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.200179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.200181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.200184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.200195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.200206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.200455Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|262bd5d5-49e112d2-a51713ba-1f228b99_71"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|262bd5d5-49e112d2-a51713ba-1f228b99_71"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 71
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.203043Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.203309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736269:3616] destroyed
2025-11-26T14:16:58.203961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736272:3617], now have 1 active actors on pipe
2025-11-26T14:16:58.204048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.204054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.204073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 71 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.204081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.204094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.204100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.204108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.204110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.204114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.204137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-26T14:16:58.204167Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 262214 count 1 nextOffset 72 batches 1
2025-11-26T14:16:58.204173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.204176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.204179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.204312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 71,1 HeadOffset 71 endOffset 71 curOffset 72 d0000000000_00000000000000000071_00000_0000000001_00000? size 262204 WTime 1764166618204
2025-11-26T14:16:58.204355Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.204368Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 71 partNo 0 count 1 size 262204
2025-11-26T14:16:58.206680Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 71 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.206696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.206756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.208501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:58.208529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.208540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.208550Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 71, partNo: 0, Offset: 71 is stored on disk
2025-11-26T14:16:58.208582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.208584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.208586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.208588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.208590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.208599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.208610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.208752Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 71
    Offset: 71
    WriteTimestampMS: 1764166618200
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.209912Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.210553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736272:3617] destroyed
2025-11-26T14:16:58.210565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736277:3620], now have 1 active actors on pipe
2025-11-26T14:16:58.210573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.210598Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.210623Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a8d4c0ef-d2fd3081-461e3712-14aa27dc_72 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.210631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.210634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.210638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.210640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.210644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.210652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.210654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.210657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.210679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.210690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.210835Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a8d4c0ef-d2fd3081-461e3712-14aa27dc_72"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a8d4c0ef-d2fd3081-461e3712-14aa27dc_72"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 72
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.212350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736277:3620] destroyed
2025-11-26T14:16:58.214799Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.215309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736280:3621], now have 1 active actors on pipe
2025-11-26T14:16:58.215799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.215809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.215831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 72 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.215842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.215854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.215857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.215865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.215866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.215870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.215897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-26T14:16:58.215936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 72 PartNo 0 PackedSize 262214 count 1 nextOffset 73 batches 1
2025-11-26T14:16:58.215943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.215945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.215947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.216052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 72,1 HeadOffset 72 endOffset 72 curOffset 73 d0000000000_00000000000000000072_00000_0000000001_00000? size 262204 WTime 1764166618215
2025-11-26T14:16:58.216087Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.216098Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 72 partNo 0 count 1 size 262204
2025-11-26T14:16:58.218499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 72 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.218516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.218537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.218548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.218556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 72, partNo: 0, Offset: 72 is stored on disk
2025-11-26T14:16:58.218598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.218603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.218604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.218607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.218609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.218616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.218626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.218727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.218820Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.219251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736280:3621] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 72
    Offset: 72
    WriteTimestampMS: 1764166618215
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.221124Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.221644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736284:3623], now have 1 active actors on pipe
2025-11-26T14:16:58.221829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.221837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.221867Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|457f7be7-7d5bdd6d-95fc24ad-b560f588_73 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.221876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.221879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.221883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.221886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.221889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.221898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.221903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.221905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.221920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.221932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.222055Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.222455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736284:3623] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|457f7be7-7d5bdd6d-95fc24ad-b560f588_73"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|457f7be7-7d5bdd6d-95fc24ad-b560f588_73"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 73
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.224795Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.225286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736287:3624], now have 1 active actors on pipe
2025-11-26T14:16:58.225762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.225775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.225804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 73 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.225815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.225830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.225833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.225842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.225844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.225848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.225873Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-26T14:16:58.225912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 73 PartNo 0 PackedSize 262214 count 1 nextOffset 74 batches 1
2025-11-26T14:16:58.225920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.225924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.225927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.226039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 73,1 HeadOffset 73 endOffset 73 curOffset 74 d0000000000_00000000000000000073_00000_0000000001_00000? size 262204 WTime 1764166618220
2025-11-26T14:16:58.226072Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.226084Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 73 partNo 0 count 1 size 262204
2025-11-26T14:16:58.228291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 73 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.228311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.228333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.228351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.228359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 73, partNo: 0, Offset: 73 is stored on disk
2025-11-26T14:16:58.228387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.228390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.228392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.228395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.228397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.228403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.228412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.228515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.228612Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.229272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736287:3624] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 73
    Offset: 73
    WriteTimestampMS: 1764166618220
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.229999Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.230602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736291:3626], now have 1 active actors on pipe
2025-11-26T14:16:58.230761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.230768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.230796Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|513cfb99-107860f0-eb231da8-5fda69db_74 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.230805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.230808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.230812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.230814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.230817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.230825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.230827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.230830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.230842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.230853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.231034Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.231423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736291:3626] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|513cfb99-107860f0-eb231da8-5fda69db_74"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|513cfb99-107860f0-eb231da8-5fda69db_74"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 74
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.233777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.234337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736294:3627], now have 1 active actors on pipe
2025-11-26T14:16:58.234824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.234835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.234855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 74 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.234865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.234877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.234880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.234890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.234891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.234895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.234920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-26T14:16:58.234951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 74 PartNo 0 PackedSize 262214 count 1 nextOffset 75 batches 1
2025-11-26T14:16:58.234958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.234961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.234965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.235089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 74,1 HeadOffset 74 endOffset 74 curOffset 75 d0000000000_00000000000000000074_00000_0000000001_00000? size 262204 WTime 1764166618234
2025-11-26T14:16:58.235126Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.235137Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 74 partNo 0 count 1 size 262204
2025-11-26T14:16:58.237425Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 74 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.237441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.237461Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.237470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.237479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 74, partNo: 0, Offset: 74 is stored on disk
2025-11-26T14:16:58.237514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.237517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.237519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.237521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.237523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.237529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.237537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.237641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.237725Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.238173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736294:3627] destroyed
2025-11-26T14:16:58.238183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.238186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.238187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.238190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.238191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 74
    Offset: 74
    WriteTimestampMS: 1764166618234
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.238873Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.239384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736298:3629], now have 1 active actors on pipe
2025-11-26T14:16:58.239550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.239560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.239586Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|23486d60-44bc05a2-838dc22d-7e744d00_75 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.239593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.239596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.239599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.239601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.239604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.239612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.239614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.239616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.239625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.239635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.239746Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.240094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736298:3629] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|23486d60-44bc05a2-838dc22d-7e744d00_75"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|23486d60-44bc05a2-838dc22d-7e744d00_75"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 75
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.242505Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.243024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736301:3630], now have 1 active actors on pipe
2025-11-26T14:16:58.243537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.243547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.243568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 75 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.243578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.243590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.243595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.243603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.243605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.243610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.243635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-26T14:16:58.243668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 75 PartNo 0 PackedSize 262214 count 1 nextOffset 76 batches 1
2025-11-26T14:16:58.243675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.243678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.243680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.243805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 75,1 HeadOffset 75 endOffset 75 curOffset 76 d0000000000_00000000000000000075_00000_0000000001_00000? size 262204 WTime 1764166618243
2025-11-26T14:16:58.243843Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.243858Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 75 partNo 0 count 1 size 262204
2025-11-26T14:16:58.245994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 75 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.246021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.246041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.246050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.246059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 75, partNo: 0, Offset: 75 is stored on disk
2025-11-26T14:16:58.246092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.246095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.246098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.246100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.246102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.246110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.246119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.246226Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.246327Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.246886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736301:3630] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 75
    Offset: 75
    WriteTimestampMS: 1764166618243
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.247514Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.247996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736305:3632], now have 1 active actors on pipe
2025-11-26T14:16:58.248151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.248158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.248184Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1dbff7b4-f04bd3d5-71e56353-c1d145e5_76 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.248191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.248194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.248198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.248200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.248203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.248211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.248214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.248216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.248229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.248241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.248360Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.248730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736305:3632] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1dbff7b4-f04bd3d5-71e56353-c1d145e5_76"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1dbff7b4-f04bd3d5-71e56353-c1d145e5_76"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 76
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.250989Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.251463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736308:3633], now have 1 active actors on pipe
2025-11-26T14:16:58.251884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.251894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.251913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 76 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.251922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.251933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.251937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.251944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.251949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.251953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.251991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-26T14:16:58.252027Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 76 PartNo 0 PackedSize 262214 count 1 nextOffset 77 batches 1
2025-11-26T14:16:58.252033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.252036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.252038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.252148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 76,1 HeadOffset 76 endOffset 76 curOffset 77 d0000000000_00000000000000000076_00000_0000000001_00000? size 262204 WTime 1764166618251
2025-11-26T14:16:58.252185Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.252197Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 76 partNo 0 count 1 size 262204
2025-11-26T14:16:58.254728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 76 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.254746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.254766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.254775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.254784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 76, partNo: 0, Offset: 76 is stored on disk
2025-11-26T14:16:58.254811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.254813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.254815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.254818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.254820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.254826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.254835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.254936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.255031Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.255570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736308:3633] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 76
    Offset: 76
    WriteTimestampMS: 1764166618251
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.256217Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.257020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736312:3635], now have 1 active actors on pipe
2025-11-26T14:16:58.257185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.257194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.257218Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7b0f5d56-7e2118e2-3080cdc7-df2c5d83_77 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.257225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.257229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.257232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.257236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.257239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.257246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.257248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.257251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.257262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.257272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.257391Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.257779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736312:3635] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7b0f5d56-7e2118e2-3080cdc7-df2c5d83_77"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7b0f5d56-7e2118e2-3080cdc7-df2c5d83_77"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 77
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.259951Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.260463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736315:3636], now have 1 active actors on pipe
2025-11-26T14:16:58.263266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.263282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.263304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 77 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.263315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.263333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.263336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.263345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.263347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.263352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.263379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-26T14:16:58.263418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 77 PartNo 0 PackedSize 262214 count 1 nextOffset 78 batches 1
2025-11-26T14:16:58.263425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.263428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.263431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.263556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 77,1 HeadOffset 77 endOffset 77 curOffset 78 d0000000000_00000000000000000077_00000_0000000001_00000? size 262204 WTime 1764166618263
2025-11-26T14:16:58.263600Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.263616Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 77 partNo 0 count 1 size 262204
2025-11-26T14:16:58.266058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 77 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.266082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.266107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.266117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.266131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 77, partNo: 0, Offset: 77 is stored on disk
2025-11-26T14:16:58.266166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.266172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.266174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.266176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.266178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.266186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.266196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.266305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.266381Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.267068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736315:3636] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 77
    Offset: 77
    WriteTimestampMS: 1764166618263
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.267780Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.268323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736319:3638], now have 1 active actors on pipe
2025-11-26T14:16:58.268510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.268520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.268548Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|11e8f3f1-6171e0d1-1b44026a-6af8a118_78 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.268557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.268562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.268566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.268568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.268571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.268581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.268584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.268588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.268601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.268615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.268750Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.269214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736319:3638] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|11e8f3f1-6171e0d1-1b44026a-6af8a118_78"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|11e8f3f1-6171e0d1-1b44026a-6af8a118_78"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 78
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.271523Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.272075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736322:3639], now have 1 active actors on pipe
2025-11-26T14:16:58.272656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.272667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.272686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 78 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.272696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.272709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.272712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.272720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.272729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.272733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.272758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-26T14:16:58.272791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 78 PartNo 0 PackedSize 262214 count 1 nextOffset 79 batches 1
2025-11-26T14:16:58.272800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.272802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.272806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.272924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 78,1 HeadOffset 78 endOffset 78 curOffset 79 d0000000000_00000000000000000078_00000_0000000001_00000? size 262204 WTime 1764166618269
2025-11-26T14:16:58.272960Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.272972Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 78 partNo 0 count 1 size 262204
2025-11-26T14:16:58.275346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 78 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.275363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.275385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.275393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.275401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 78, partNo: 0, Offset: 78 is stored on disk
2025-11-26T14:16:58.275432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.275434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.275437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.275438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.275440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.275455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.275464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.275566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.275658Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.276168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736322:3639] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 78
    Offset: 78
    WriteTimestampMS: 1764166618269
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.276810Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.277420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736326:3641], now have 1 active actors on pipe
2025-11-26T14:16:58.277433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.277437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.277461Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2b490dd9-2e0c5554-b812b535-e31b4393_79 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.277469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.277472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.277475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.277477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.277479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.277487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.277489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.277501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.277511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.277521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.277663Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.278058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736326:3641] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2b490dd9-2e0c5554-b812b535-e31b4393_79"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2b490dd9-2e0c5554-b812b535-e31b4393_79"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 79
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.280267Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.280912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736329:3642], now have 1 active actors on pipe
2025-11-26T14:16:58.281370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.281379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.281406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 79 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.281415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.281427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.281430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.281437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.281438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.281442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.281465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-26T14:16:58.281496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 79 PartNo 0 PackedSize 262214 count 1 nextOffset 80 batches 1
2025-11-26T14:16:58.281503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.281505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.281507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.281618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 79,1 HeadOffset 79 endOffset 79 curOffset 80 d0000000000_00000000000000000079_00000_0000000001_00000? size 262204 WTime 1764166618279
2025-11-26T14:16:58.281654Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.281665Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 79 partNo 0 count 1 size 262204
2025-11-26T14:16:58.283684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 79 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.283702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.283772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.284489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:58.284538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.284550Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.284565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 79, partNo: 0, Offset: 79 is stored on disk
2025-11-26T14:16:58.284624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.284627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.284632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.284634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.284636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.284647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.284673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.284802Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.285356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736329:3642] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 79
    Offset: 79
    WriteTimestampMS: 1764166618279
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.289138Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.289684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736333:3644], now have 1 active actors on pipe
2025-11-26T14:16:58.289889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.289898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.289925Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|baa825c0-23dd28e2-27baa4a4-c94cb985_80 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.289935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.289938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.289942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.289945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.289949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.289958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.289960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.289964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.289979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.289992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.290125Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.290526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736333:3644] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|baa825c0-23dd28e2-27baa4a4-c94cb985_80"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|baa825c0-23dd28e2-27baa4a4-c94cb985_80"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 80
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.292847Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.293736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736337:3645], now have 1 active actors on pipe
2025-11-26T14:16:58.293831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.293837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.293856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 80 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.293866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.293879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.293887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.293896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.293899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.293903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.293929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-26T14:16:58.293964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 80 PartNo 0 PackedSize 262214 count 1 nextOffset 81 batches 1
2025-11-26T14:16:58.293973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.293975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.293981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.294092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 80,1 HeadOffset 80 endOffset 80 curOffset 81 d0000000000_00000000000000000080_00000_0000000001_00000? size 262204 WTime 1764166618292
2025-11-26T14:16:58.294128Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.294157Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 80 partNo 0 count 1 size 262204
2025-11-26T14:16:58.296910Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 80 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.296930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.296959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.296967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.296975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 80, partNo: 0, Offset: 80 is stored on disk
2025-11-26T14:16:58.297005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.297007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.297010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.297012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.297014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.297021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.297030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.297144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.297269Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.297836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736337:3645] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 80
    Offset: 80
    WriteTimestampMS: 1764166618292
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.298505Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.299613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736341:3648], now have 1 active actors on pipe
2025-11-26T14:16:58.299628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.299634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.299660Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cf7e4c77-fe18ecfc-c7e8509e-53c90028_81 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.299670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.299675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.299679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.299681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.299684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.299692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.299695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.299697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.299711Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.299726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.299904Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.300404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736341:3648] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cf7e4c77-fe18ecfc-c7e8509e-53c90028_81"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cf7e4c77-fe18ecfc-c7e8509e-53c90028_81"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 81
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.302949Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.303702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736344:3649], now have 1 active actors on pipe
2025-11-26T14:16:58.304243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.304256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.304289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 81 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.304299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.304313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.304317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.304326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.304328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.304332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.304358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-26T14:16:58.304404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 81 PartNo 0 PackedSize 262214 count 1 nextOffset 82 batches 1
2025-11-26T14:16:58.304410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.304413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.304416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.304539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 81,1 HeadOffset 81 endOffset 81 curOffset 82 d0000000000_00000000000000000081_00000_0000000001_00000? size 262204 WTime 1764166618304
2025-11-26T14:16:58.304576Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.304588Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 81 partNo 0 count 1 size 262204
2025-11-26T14:16:58.307011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 81 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.307031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.307053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.307062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.307074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 81, partNo: 0, Offset: 81 is stored on disk
2025-11-26T14:16:58.307105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.307108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.307109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.307112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.307113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.307121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.307130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.307288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.307389Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.308087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736344:3649] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 81
    Offset: 81
    WriteTimestampMS: 1764166618300
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.312908Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.320831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736348:3651], now have 1 active actors on pipe
2025-11-26T14:16:58.321081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.321100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.321136Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e48b83c2-1dd2939-3f1250b0-89cd25a1_82 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.321149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.321153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.321159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.321162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.321169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.321180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.321183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.321186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.321202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.321220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e48b83c2-1dd2939-3f1250b0-89cd25a1_82"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e48b83c2-1dd2939-3f1250b0-89cd25a1_82"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 82
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.321604Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.323679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736348:3651] destroyed
2025-11-26T14:16:58.325715Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.326245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736355:3654], now have 1 active actors on pipe
2025-11-26T14:16:58.326793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.326805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.326830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 82 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.326842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.326857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.326860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.326871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.326874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.326877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.326918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-26T14:16:58.326955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 82 PartNo 0 PackedSize 262214 count 1 nextOffset 83 batches 1
2025-11-26T14:16:58.326962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.326965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.326969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.327096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 82,1 HeadOffset 82 endOffset 82 curOffset 83 d0000000000_00000000000000000082_00000_0000000001_00000? size 262204 WTime 1764166618325
2025-11-26T14:16:58.327144Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.327160Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 82 partNo 0 count 1 size 262204
2025-11-26T14:16:58.339496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-26T14:16:58.340792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 82 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.340820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.340845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.340856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.340867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 82, partNo: 0, Offset: 82 is stored on disk
2025-11-26T14:16:58.340923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.340927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.340929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.340932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.340935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.340947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.340960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.341082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.341163Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 82
    Offset: 82
    WriteTimestampMS: 1764166618325
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 14
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.342196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736355:3654] destroyed
2025-11-26T14:16:58.344613Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.351021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736368:3661], now have 1 active actors on pipe
2025-11-26T14:16:58.356367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.356388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.356426Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|24e66a9f-80f6c2a6-61bd615f-6a75f2a8_83 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.356437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.356441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.356447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.356450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.356455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.356466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.356469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.356473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.356502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.356518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.360481Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|24e66a9f-80f6c2a6-61bd615f-6a75f2a8_83"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|24e66a9f-80f6c2a6-61bd615f-6a75f2a8_83"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 83
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.364320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736368:3661] destroyed
2025-11-26T14:16:58.368367Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.372876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736371:3662], now have 1 active actors on pipe
2025-11-26T14:16:58.373479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.373493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.373515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 83 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.373528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.373547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.373552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.373560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.373563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.373568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.373597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-26T14:16:58.373635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 83 PartNo 0 PackedSize 262214 count 1 nextOffset 84 batches 1
2025-11-26T14:16:58.373642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.373647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.373650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.373774Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 83,1 HeadOffset 83 endOffset 83 curOffset 84 d0000000000_00000000000000000083_00000_0000000001_00000? size 262204 WTime 1764166618373
2025-11-26T14:16:58.373813Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.373827Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 83 partNo 0 count 1 size 262204
2025-11-26T14:16:58.384922Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 83 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.384951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.384978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.384989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.385001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 83, partNo: 0, Offset: 83 is stored on disk
2025-11-26T14:16:58.385040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.385047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.385049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.385052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.385054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.385065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.385078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.385222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 83
    Offset: 83
    WriteTimestampMS: 1764166618373
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.387211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736371:3662] destroyed
2025-11-26T14:16:58.387227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736375:3664], now have 1 active actors on pipe
2025-11-26T14:16:58.387239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.387246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.387291Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|33f46a07-9e35828-55ca9ad0-ac611c9b_84 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.387301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.387304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.387308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.387311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.387315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.387326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.387329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.387334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.387347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.387358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|33f46a07-9e35828-55ca9ad0-ac611c9b_84"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:58.385337Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.386512Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.387556Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|33f46a07-9e35828-55ca9ad0-ac611c9b_84"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 84
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.390396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736375:3664] destroyed
2025-11-26T14:16:58.390316Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.395902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736378:3665], now have 1 active actors on pipe
2025-11-26T14:16:58.396030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.396039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.396063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 84 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.396074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.396091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.396095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.396104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.396107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.396113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.396144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-26T14:16:58.396189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 84 PartNo 0 PackedSize 262214 count 1 nextOffset 85 batches 1
2025-11-26T14:16:58.396196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.396199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.396205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.396503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 84,1 HeadOffset 84 endOffset 84 curOffset 85 d0000000000_00000000000000000084_00000_0000000001_00000? size 262204 WTime 1764166618396
2025-11-26T14:16:58.396553Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.396567Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 84 partNo 0 count 1 size 262204
2025-11-26T14:16:58.412752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 84 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.412781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.412808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.412820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.412830Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 84, partNo: 0, Offset: 84 is stored on disk
2025-11-26T14:16:58.412872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.412876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.412879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.412883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.412885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.412895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.412908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 84
    Offset: 84
    WriteTimestampMS: 1764166618396
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 16
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.413053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.415036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736378:3665] destroyed
2025-11-26T14:16:58.415050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736382:3667], now have 1 active actors on pipe
2025-11-26T14:16:58.415063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.415068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.415106Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bcbb2f6c-952cb997-6c9049a4-ff3d27d9_85 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.415114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.415118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.415123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.415125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.415129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.415137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.415139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.415143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.415156Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.415168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bcbb2f6c-952cb997-6c9049a4-ff3d27d9_85"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:58.413140Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.414315Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.415381Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.416768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736382:3667] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bcbb2f6c-952cb997-6c9049a4-ff3d27d9_85"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 85
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.418861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736385:3668], now have 1 active actors on pipe
2025-11-26T14:16:58.419367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.419379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.419401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 85 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.419414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.419427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.419431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.419439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.419441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.419446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.419471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-26T14:16:58.419507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 85 PartNo 0 PackedSize 262214 count 1 nextOffset 86 batches 1
2025-11-26T14:16:58.419512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.419515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.419518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.419638Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 85,1 HeadOffset 85 endOffset 85 curOffset 86 d0000000000_00000000000000000085_00000_0000000001_00000? size 262204 WTime 1764166618419
2025-11-26T14:16:58.419681Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.419693Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 85 partNo 0 count 1 size 262204
2025-11-26T14:16:58.418330Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.421967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 85 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.421988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.422011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.422019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.422028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 85, partNo: 0, Offset: 85 is stored on disk
2025-11-26T14:16:58.422062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.422065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.422067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.422069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.422071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.422078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.422088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.422232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 85
    Offset: 85
    WriteTimestampMS: 1764166618419
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.423794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736385:3668] destroyed
2025-11-26T14:16:58.422370Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.423557Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.424307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736389:3670], now have 1 active actors on pipe
2025-11-26T14:16:58.424321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.424327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.424354Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a4b00b29-6062608a-f9f31ba9-9d49b059_86 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.424363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.424367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.424371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.424374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.424378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.424386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.424391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.424394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.424408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.424421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a4b00b29-6062608a-f9f31ba9-9d49b059_86"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a4b00b29-6062608a-f9f31ba9-9d49b059_86"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 86
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.427237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736389:3670] destroyed
2025-11-26T14:16:58.427827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736392:3671], now have 1 active actors on pipe
2025-11-26T14:16:58.424566Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.427320Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.428356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.428367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.428389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 86 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.428400Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.428414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.428418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.428426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.428429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.428434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.428462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-26T14:16:58.428499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 86 PartNo 0 PackedSize 262214 count 1 nextOffset 87 batches 1
2025-11-26T14:16:58.428506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.428509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.428512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.428644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 86,1 HeadOffset 86 endOffset 86 curOffset 87 d0000000000_00000000000000000086_00000_0000000001_00000? size 262204 WTime 1764166618428
2025-11-26T14:16:58.428685Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.428697Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 86 partNo 0 count 1 size 262204
2025-11-26T14:16:58.430971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 86 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.430990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.431057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.432950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:58.432990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.433004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.433037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 86, partNo: 0, Offset: 86 is stored on disk
2025-11-26T14:16:58.433090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.433095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.433097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.433100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.433102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.433115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.433135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 86
    Offset: 86
    WriteTimestampMS: 1764166618422
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 6
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.433409Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.435223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736392:3671] destroyed
2025-11-26T14:16:58.435713Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.436336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736396:3673], now have 1 active actors on pipe
2025-11-26T14:16:58.436657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.436666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.436706Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9eb6ac2c-896f944b-cee5915-1406214b_87 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.436716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.436720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.436725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.436727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.436731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.436740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.436743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.436746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.436762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.436777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.436944Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.437485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736396:3673] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9eb6ac2c-896f944b-cee5915-1406214b_87"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9eb6ac2c-896f944b-cee5915-1406214b_87"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 87
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.440480Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.443410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.443426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.443429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.443432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.443435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.443458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736403:3675], now have 1 active actors on pipe
2025-11-26T14:16:58.443593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.443605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.443628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 87 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.443638Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.443651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.443654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.443663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.443665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.443669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.443701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-26T14:16:58.443737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 87 PartNo 0 PackedSize 262214 count 1 nextOffset 88 batches 1
2025-11-26T14:16:58.443745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.443749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.443752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.443866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 87,1 HeadOffset 87 endOffset 87 curOffset 88 d0000000000_00000000000000000087_00000_0000000001_00000? size 262204 WTime 1764166618441
2025-11-26T14:16:58.443905Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.443918Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 87 partNo 0 count 1 size 262204
2025-11-26T14:16:58.446288Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 87 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.446312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.446332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.446341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.446355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 87, partNo: 0, Offset: 87 is stored on disk
2025-11-26T14:16:58.446391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.446398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.446400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.446403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.446405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.446413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.446429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.446486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.446566Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 87
    Offset: 87
    WriteTimestampMS: 1764166618441
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:58.447015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736403:3675] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.447555Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.448030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736432:3686], now have 1 active actors on pipe
2025-11-26T14:16:58.448124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.448137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.448168Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c557908a-ade3d9a8-503de7e0-f111c5ec_88 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.448180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.448188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.448191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.448194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.448198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.448206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.448209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.448211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.448223Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.448235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.448336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c557908a-ade3d9a8-503de7e0-f111c5ec_88"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c557908a-ade3d9a8-503de7e0-f111c5ec_88"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 88
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.449453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736432:3686] destroyed
2025-11-26T14:16:58.453530Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.454501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736435:3687], now have 1 active actors on pipe
2025-11-26T14:16:58.454629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.454641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.454670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 88 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.454685Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.454701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.454708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.454718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.454724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.454730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.454756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-26T14:16:58.454797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 88 PartNo 0 PackedSize 262214 count 1 nextOffset 89 batches 1
2025-11-26T14:16:58.454804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.454807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.454809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.454944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 88,1 HeadOffset 88 endOffset 88 curOffset 89 d0000000000_00000000000000000088_00000_0000000001_00000? size 262204 WTime 1764166618452
2025-11-26T14:16:58.454985Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.454998Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 88 partNo 0 count 1 size 262204
2025-11-26T14:16:58.465135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 88 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.465174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.465203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.465216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.465229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 88, partNo: 0, Offset: 88 is stored on disk
2025-11-26T14:16:58.465274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.465278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.465280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.465283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.465285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.465296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.465308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.465528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.465643Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.466132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736435:3687] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 88
    Offset: 88
    WriteTimestampMS: 1764166618452
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.472984Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.473681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736441:3689], now have 1 active actors on pipe
2025-11-26T14:16:58.473737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.473746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.473787Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bb7ad4d5-bc01ad37-aca87dd0-fb64cad_89 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.473804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.473811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.473817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.473822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.473827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.473840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.473843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.473847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.473865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.473885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.473960Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.474348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736441:3689] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bb7ad4d5-bc01ad37-aca87dd0-fb64cad_89"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bb7ad4d5-bc01ad37-aca87dd0-fb64cad_89"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 89
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.485575Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.486186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736444:3691], now have 1 active actors on pipe
2025-11-26T14:16:58.486769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.486788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.486812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 89 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.486845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.486867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.486872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.486886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.486893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.486898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.486929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-26T14:16:58.486973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 89 PartNo 0 PackedSize 262214 count 1 nextOffset 90 batches 1
2025-11-26T14:16:58.486985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.486988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.486991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.487142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 89,1 HeadOffset 89 endOffset 89 curOffset 90 d0000000000_00000000000000000089_00000_0000000001_00000? size 262204 WTime 1764166618481
2025-11-26T14:16:58.487194Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.487212Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 89 partNo 0 count 1 size 262204
2025-11-26T14:16:58.497092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 89 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.497128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.497166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.497178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.497191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 89, partNo: 0, Offset: 89 is stored on disk
2025-11-26T14:16:58.497234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.497238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.497240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.497245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.497247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.497258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.497273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.498478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.498649Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.499205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736444:3691] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 89
    Offset: 89
    WriteTimestampMS: 1764166618481
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 15
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.508918Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.512555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736448:3693], now have 1 active actors on pipe
2025-11-26T14:16:58.512820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.512832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.512870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e79fc7ee-57f20ec2-2f8ebc2d-db2dc3ac_90 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.512882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.512886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.512893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.512895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.512900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.512912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.512915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.512919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.512936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.512959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.513148Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.513660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736448:3693] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e79fc7ee-57f20ec2-2f8ebc2d-db2dc3ac_90"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e79fc7ee-57f20ec2-2f8ebc2d-db2dc3ac_90"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 90
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.524397Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.525173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736451:3694], now have 1 active actors on pipe
2025-11-26T14:16:58.525803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.525819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.525844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 90 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.525857Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.525873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.525878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.525888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.525891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.525896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.525928Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-26T14:16:58.525973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 90 PartNo 0 PackedSize 262214 count 1 nextOffset 91 batches 1
2025-11-26T14:16:58.525982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.525985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.525989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.526116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 90,1 HeadOffset 90 endOffset 90 curOffset 91 d0000000000_00000000000000000090_00000_0000000001_00000? size 262204 WTime 1764166618525
2025-11-26T14:16:58.526159Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.526173Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 90 partNo 0 count 1 size 262204
2025-11-26T14:16:58.528720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 90 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.528743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.528766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.528777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.528786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 90, partNo: 0, Offset: 90 is stored on disk
2025-11-26T14:16:58.528829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.528832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.528835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.528837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.528839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.528850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.528861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.528988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.529077Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.529675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736451:3694] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 90
    Offset: 90
    WriteTimestampMS: 1764166618525
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.534664Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.544498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.544517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.544519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.544528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.544531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.544811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736455:3696], now have 1 active actors on pipe
2025-11-26T14:16:58.545029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.545039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.545070Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bcce6a8c-2e29d550-a86cfe7-d0f7df6f_91 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.545078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.545082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.545087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.545089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.545094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.545103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.545107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.545110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.545123Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.545142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.545289Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.545740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736455:3696] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bcce6a8c-2e29d550-a86cfe7-d0f7df6f_91"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bcce6a8c-2e29d550-a86cfe7-d0f7df6f_91"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 91
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.552375Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.553162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736458:3697], now have 1 active actors on pipe
2025-11-26T14:16:58.553708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.553722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.553744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 91 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.553757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.553772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.553776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.553786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.553788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.553794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.553822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-26T14:16:58.553878Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 91 PartNo 0 PackedSize 262214 count 1 nextOffset 92 batches 1
2025-11-26T14:16:58.553885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.553888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.553891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.554015Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 91,1 HeadOffset 91 endOffset 91 curOffset 92 d0000000000_00000000000000000091_00000_0000000001_00000? size 262204 WTime 1764166618553
2025-11-26T14:16:58.554053Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.554067Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 91 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 91
    Offset: 91
    WriteTimestampMS: 1764166618553
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-26T14:16:58.557747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 91 count 1 size 262204 actorID [2:7577037922626401473:2365]
CallPersQueueGRPC request to localhost:28333
2025-11-26T14:16:58.557777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.557804Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.557815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.557825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 91, partNo: 0, Offset: 91 is stored on disk
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.557862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.557866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.557868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.557870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.558024Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.557872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.557881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.557891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.557953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.558731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736458:3697] destroyed
2025-11-26T14:16:58.564767Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.565378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736462:3699], now have 1 active actors on pipe
2025-11-26T14:16:58.565583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.565593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.565628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7b9532e0-eee5d673-9deb89d9-3b6fabf7_92 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.565639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.565643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.565652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.565655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.565659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.565668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.565671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.565675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.565693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.565708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.565857Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.566254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736462:3699] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7b9532e0-eee5d673-9deb89d9-3b6fabf7_92"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7b9532e0-eee5d673-9deb89d9-3b6fabf7_92"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 92
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.569105Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.569683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736465:3700], now have 1 active actors on pipe
2025-11-26T14:16:58.570192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.570205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.570227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 92 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.570237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.570265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.570268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.570279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.570282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.570288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.570315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-26T14:16:58.570351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 92 PartNo 0 PackedSize 262214 count 1 nextOffset 93 batches 1
2025-11-26T14:16:58.570359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.570362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.570364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.570485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 92,1 HeadOffset 92 endOffset 92 curOffset 93 d0000000000_00000000000000000092_00000_0000000001_00000? size 262204 WTime 1764166618570
2025-11-26T14:16:58.570525Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.570538Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 92 partNo 0 count 1 size 262204
2025-11-26T14:16:58.573232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 92 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.573256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.573278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.573288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.573298Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 92, partNo: 0, Offset: 92 is stored on disk
2025-11-26T14:16:58.573332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.573336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.573338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.573341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.573344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.573354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.573364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.573498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.573578Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.574199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736465:3700] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 92
    Offset: 92
    WriteTimestampMS: 1764166618569
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.574911Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.575579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736469:3702], now have 1 active actors on pipe
2025-11-26T14:16:58.575592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.575600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.575629Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b50e4e70-eeaf092-52b8e163-658a8897_93 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.575638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.575641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.575646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.575648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.575652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.575661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.575663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.575667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.575680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.575691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.575836Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.576285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736469:3702] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b50e4e70-eeaf092-52b8e163-658a8897_93"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b50e4e70-eeaf092-52b8e163-658a8897_93"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 93
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.578707Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.579232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736472:3703], now have 1 active actors on pipe
2025-11-26T14:16:58.579730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.579741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.579762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 93 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.579772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.579786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.579790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.579799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.579802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.579807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.579833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-26T14:16:58.579882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 93 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-26T14:16:58.579889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.579892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.579895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.580014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,1 HeadOffset 93 endOffset 93 curOffset 94 d0000000000_00000000000000000093_00000_0000000001_00000? size 262204 WTime 1764166618575
2025-11-26T14:16:58.580051Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.580066Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 1 size 262204
2025-11-26T14:16:58.582250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.582268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.582335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.592474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-26T14:16:58.592518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.592532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.592549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 93, partNo: 0, Offset: 93 is stored on disk
2025-11-26T14:16:58.592601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.592605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.592608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.592611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.592614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.592627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.592644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.592921Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 93
    Offset: 93
    WriteTimestampMS: 1764166618575
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 15
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.594434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736472:3703] destroyed
2025-11-26T14:16:58.594523Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.595247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736476:3705], now have 1 active actors on pipe
2025-11-26T14:16:58.595262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.595271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.595305Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7a0723-ac60f8ed-e3a86911-56d5d848_94 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.595316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.595322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.595327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.595330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.595334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.595344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.595346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.595349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.595367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.595381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.595532Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7a0723-ac60f8ed-e3a86911-56d5d848_94"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7a0723-ac60f8ed-e3a86911-56d5d848_94"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 94
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.598591Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.598699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736476:3705] destroyed
2025-11-26T14:16:58.600085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736479:3706], now have 1 active actors on pipe
2025-11-26T14:16:58.600202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.600211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.600237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 94 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.600248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.600282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.600288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.600300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.600302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.600307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.600337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-26T14:16:58.600371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 94 PartNo 0 PackedSize 262214 count 1 nextOffset 95 batches 1
2025-11-26T14:16:58.600378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.600382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.600385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.600512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 94,1 HeadOffset 94 endOffset 94 curOffset 95 d0000000000_00000000000000000094_00000_0000000001_00000? size 262204 WTime 1764166618600
2025-11-26T14:16:58.600550Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.600564Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 94 partNo 0 count 1 size 262204
2025-11-26T14:16:58.603451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 94 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.603474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.603496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.603508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.603518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 94, partNo: 0, Offset: 94 is stored on disk
2025-11-26T14:16:58.603560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.603563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.603566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.603570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.603573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.603582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.603593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.603664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.603837Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 94
    Offset: 94
    WriteTimestampMS: 1764166618595
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 4
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-26T14:16:58.605336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.606145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736479:3706] destroyed
2025-11-26T14:16:58.606161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736483:3708], now have 1 active actors on pipe
2025-11-26T14:16:58.606173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.606180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.606213Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a4fea088-afd3ad59-774ee22-cf5da80_95 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-26T14:16:58.606224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.606228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.606233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.606236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.606240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.606250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.606254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.606257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.606274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-26T14:16:58.606290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.606520Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a4fea088-afd3ad59-774ee22-cf5da80_95"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a4fea088-afd3ad59-774ee22-cf5da80_95"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 95
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.610310Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.611550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736483:3708] destroyed
2025-11-26T14:16:58.611585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736488:3709], now have 1 active actors on pipe
2025-11-26T14:16:58.611696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.611705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.611729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2008: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 95 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-26T14:16:58.611744Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-26T14:16:58.611758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.611762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-26T14:16:58.611771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-26T14:16:58.611774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.611780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-26T14:16:58.611810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-26T14:16:58.611849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 95 PartNo 0 PackedSize 262214 count 1 nextOffset 96 batches 1
2025-11-26T14:16:58.611856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-26T14:16:58.611859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-26T14:16:58.611862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.611993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 95,1 HeadOffset 95 endOffset 95 curOffset 96 d0000000000_00000000000000000095_00000_0000000001_00000? size 262204 WTime 1764166618609
2025-11-26T14:16:58.612034Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.612048Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 95 partNo 0 count 1 size 262204
2025-11-26T14:16:58.614709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 95 count 1 size 262204 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.614736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.614760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.614771Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-26T14:16:58.614782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 95, partNo: 0, Offset: 95 is stored on disk
2025-11-26T14:16:58.614819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.614823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.614826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.614829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.614831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.614842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000064_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000065_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000066_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000067_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000068_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000069_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000070_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000071_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000072_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000073_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000074_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000075_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000076_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000077_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000078_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000079_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000080_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000081_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000082_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000083_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000084_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000085_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000086_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000087_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000088_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000089_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000090_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000091_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000092_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000093_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000094_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000095_00000_0000000001_00000? (262204)
2025-11-26T14:16:58.614940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-26T14:16:58.614953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.614968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-26T14:16:58.614971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-26T14:16:58.614981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-26T14:16:58.614989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-26T14:16:58.615000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-26T14:16:58.615009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-26T14:16:58.615030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-26T14:16:58.615041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-26T14:16:58.615049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-26T14:16:58.615058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-26T14:16:58.615085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-26T14:16:58.615094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-26T14:16:58.615103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-26T14:16:58.615112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-26T14:16:58.615120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-26T14:16:58.615129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-26T14:16:58.615138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-26T14:16:58.615147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-26T14:16:58.615190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-26T14:16:58.615199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-26T14:16:58.615221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-26T14:16:58.615230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-26T14:16:58.615240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-26T14:16:58.615250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-26T14:16:58.615260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-26T14:16:58.615269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-26T14:16:58.615279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-26T14:16:58.615288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-26T14:16:58.615297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-26T14:16:58.615307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-26T14:16:58.615316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-26T14:16:58.615326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-26T14:16:58.615335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-26T14:16:58.615345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-26T14:16:58.615358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 64 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 65 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615368Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 66 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 67 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 68 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 69 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 70 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615387Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 71 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615391Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 72 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 73 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 74 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 75 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615417Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 76 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 77 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 78 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 79 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 80 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 81 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 82 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 83 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 84 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615453Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 85 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 86 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 87 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 88 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 89 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 90 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 91 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 92 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 93 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 94 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 95 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.615503Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-26T14:16:58.615538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-26T14:16:58.615548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-26T14:16:58.615551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.615554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000064_00000_0000000001_00000?, parameters.CurOffset=64
2025-11-26T14:16:58.615644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 64.0/1
2025-11-26T14:16:58.615652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-26T14:16:58.615675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-26T14:16:58.615678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 64:0 appended
2025-11-26T14:16:58.615683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-26T14:16:58.615684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.615687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000065_00000_0000000001_00000?, parameters.CurOffset=65
2025-11-26T14:16:58.615939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 65.0/1
2025-11-26T14:16:58.615944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-26T14:16:58.616752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 524387 count 2 nextOffset 66 batches 1
2025-11-26T14:16:58.616759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 65:0 appended
2025-11-26T14:16:58.616764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-26T14:16:58.616767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.616771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000066_00000_0000000001_00000?, parameters.CurOffset=66
2025-11-26T14:16:58.617442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 66.0/1
2025-11-26T14:16:58.617452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-26T14:16:58.617473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 786601 count 3 nextOffset 67 batches 2
2025-11-26T14:16:58.617476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 66:0 appended
2025-11-26T14:16:58.617481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-26T14:16:58.617485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.617489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000067_00000_0000000001_00000?, parameters.CurOffset=67
2025-11-26T14:16:58.617556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 67.0/1
2025-11-26T14:16:58.617560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-26T14:16:58.617720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1048774 count 4 nextOffset 68 batches 2
2025-11-26T14:16:58.617723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 67:0 appended
2025-11-26T14:16:58.617728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-26T14:16:58.617729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.617732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000068_00000_0000000001_00000?, parameters.CurOffset=68
2025-11-26T14:16:58.617802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 68.0/1
2025-11-26T14:16:58.617806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-26T14:16:58.617817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1310988 count 5 nextOffset 69 batches 3
2025-11-26T14:16:58.617819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 68:0 appended
2025-11-26T14:16:58.617823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-26T14:16:58.617825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.617828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000069_00000_0000000001_00000?, parameters.CurOffset=69
2025-11-26T14:16:58.617915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 69.0/1
2025-11-26T14:16:58.617919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-26T14:16:58.618847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1573161 count 6 nextOffset 70 batches 3
2025-11-26T14:16:58.618852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 69:0 appended
2025-11-26T14:16:58.618857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-26T14:16:58.618859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.618863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000070_00000_0000000001_00000?, parameters.CurOffset=70
2025-11-26T14:16:58.619687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.619707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619768Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619771Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619791Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619799Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619818Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619829Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.619837Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63'
2025-11-26T14:16:58.620199Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.621224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 70.0/1
2025-11-26T14:16:58.621278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-26T14:16:58.621304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1835375 count 7 nextOffset 71 batches 4
2025-11-26T14:16:58.621307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 70:0 appended
2025-11-26T14:16:58.621314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-26T14:16:58.621317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.621320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000071_00000_0000000001_00000?, parameters.CurOffset=71
2025-11-26T14:16:58.621413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 71.0/1
2025-11-26T14:16:58.621420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-26T14:16:58.621632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2097548 count 8 nextOffset 72 batches 4
2025-11-26T14:16:58.621636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 71:0 appended
2025-11-26T14:16:58.621641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-26T14:16:58.621643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.621646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000072_00000_0000000001_00000?, parameters.CurOffset=72
2025-11-26T14:16:58.621723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 72.0/1
2025-11-26T14:16:58.621726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-26T14:16:58.621735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2359762 count 9 nextOffset 73 batches 5
2025-11-26T14:16:58.621736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 72:0 appended
2025-11-26T14:16:58.621739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-26T14:16:58.621741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.621742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000073_00000_0000000001_00000?, parameters.CurOffset=73
2025-11-26T14:16:58.621796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 73.0/1
2025-11-26T14:16:58.621820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-26T14:16:58.622413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2621935 count 10 nextOffset 74 batches 5
2025-11-26T14:16:58.622418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 73:0 appended
2025-11-26T14:16:58.622426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-26T14:16:58.622427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.622430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000074_00000_0000000001_00000?, parameters.CurOffset=74
2025-11-26T14:16:58.622498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 74.0/1
2025-11-26T14:16:58.622502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-26T14:16:58.622529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2884149 count 11 nextOffset 75 batches 6
2025-11-26T14:16:58.622531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 74:0 appended
2025-11-26T14:16:58.622535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-26T14:16:58.622537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.622540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000075_00000_0000000001_00000?, parameters.CurOffset=75
2025-11-26T14:16:58.622618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 75.0/1
2025-11-26T14:16:58.622622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-26T14:16:58.623251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3146322 count 12 nextOffset 76 batches 6
2025-11-26T14:16:58.623254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 75:0 appended
2025-11-26T14:16:58.623260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-26T14:16:58.623263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.623266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000076_00000_0000000001_00000?, parameters.CurOffset=76
2025-11-26T14:16:58.623336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 76.0/1
2025-11-26T14:16:58.623341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-26T14:16:58.623353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3408536 count 13 nextOffset 77 batches 7
2025-11-26T14:16:58.623357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 76:0 appended
2025-11-26T14:16:58.623361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-26T14:16:58.623363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.623365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000077_00000_0000000001_00000?, parameters.CurOffset=77
2025-11-26T14:16:58.623418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 77.0/1
2025-11-26T14:16:58.623422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-26T14:16:58.624109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3670709 count 14 nextOffset 78 batches 7
2025-11-26T14:16:58.624113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 77:0 appended
2025-11-26T14:16:58.624118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-26T14:16:58.624120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.624123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000078_00000_0000000001_00000?, parameters.CurOffset=78
2025-11-26T14:16:58.624186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 78.0/1
2025-11-26T14:16:58.624191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-26T14:16:58.624203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3932923 count 15 nextOffset 79 batches 8
2025-11-26T14:16:58.624206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 78:0 appended
2025-11-26T14:16:58.624210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-26T14:16:58.624212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.624216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000079_00000_0000000001_00000?, parameters.CurOffset=79
2025-11-26T14:16:58.624358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 79.0/1
2025-11-26T14:16:58.624367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 95
    Offset: 95
    WriteTimestampMS: 1764166618609
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-26T14:16:58.625027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4195096 count 16 nextOffset 80 batches 8
2025-11-26T14:16:58.625041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 79:0 appended
2025-11-26T14:16:58.625048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-26T14:16:58.625050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.625053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000080_00000_0000000001_00000?, parameters.CurOffset=80
2025-11-26T14:16:58.625153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 80.0/1
2025-11-26T14:16:58.625157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-26T14:16:58.625172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4457310 count 17 nextOffset 81 batches 9
2025-11-26T14:16:58.625175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 80:0 appended
2025-11-26T14:16:58.625179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-26T14:16:58.625180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.625183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000081_00000_0000000001_00000?, parameters.CurOffset=81
2025-11-26T14:16:58.625242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 81.0/1
2025-11-26T14:16:58.625246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-26T14:16:58.626037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4719483 count 18 nextOffset 82 batches 9
2025-11-26T14:16:58.626050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 81:0 appended
2025-11-26T14:16:58.626057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-26T14:16:58.626059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.626065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000082_00000_0000000001_00000?, parameters.CurOffset=82
2025-11-26T14:16:58.626140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 82.0/1
2025-11-26T14:16:58.626144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-26T14:16:58.626155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4981697 count 19 nextOffset 83 batches 10
2025-11-26T14:16:58.626157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 82:0 appended
2025-11-26T14:16:58.626160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-26T14:16:58.626163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.626166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000083_00000_0000000001_00000?, parameters.CurOffset=83
2025-11-26T14:16:58.626215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 83.0/1
2025-11-26T14:16:58.626218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-26T14:16:58.626864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5243870 count 20 nextOffset 84 batches 10
2025-11-26T14:16:58.626868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 83:0 appended
2025-11-26T14:16:58.626873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-26T14:16:58.626876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.626879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000084_00000_0000000001_00000?, parameters.CurOffset=84
2025-11-26T14:16:58.626943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 84.0/1
2025-11-26T14:16:58.626946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-26T14:16:58.626957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5506084 count 21 nextOffset 85 batches 11
2025-11-26T14:16:58.626959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 84:0 appended
2025-11-26T14:16:58.626963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-26T14:16:58.626965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.626968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000085_00000_0000000001_00000?, parameters.CurOffset=85
2025-11-26T14:16:58.627019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 85.0/1
2025-11-26T14:16:58.627022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-26T14:16:58.627558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5768257 count 22 nextOffset 86 batches 11
2025-11-26T14:16:58.627562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 85:0 appended
2025-11-26T14:16:58.627568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-26T14:16:58.627570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.627573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000086_00000_0000000001_00000?, parameters.CurOffset=86
2025-11-26T14:16:58.627636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 86.0/1
2025-11-26T14:16:58.627640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-26T14:16:58.627650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6030471 count 23 nextOffset 87 batches 12
2025-11-26T14:16:58.627652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 86:0 appended
2025-11-26T14:16:58.627656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-26T14:16:58.627658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.627661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000087_00000_0000000001_00000?, parameters.CurOffset=87
2025-11-26T14:16:58.627711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 87.0/1
2025-11-26T14:16:58.627714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-26T14:16:58.628225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6292644 count 24 nextOffset 88 batches 12
2025-11-26T14:16:58.628228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 87:0 appended
2025-11-26T14:16:58.628232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-26T14:16:58.628235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.628237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000088_00000_0000000001_00000?, parameters.CurOffset=88
2025-11-26T14:16:58.628989Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.628597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 88.0/1
2025-11-26T14:16:58.628608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-26T14:16:58.628621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6554858 count 25 nextOffset 89 batches 13
2025-11-26T14:16:58.628623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 88:0 appended
2025-11-26T14:16:58.628628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-26T14:16:58.628631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.628634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000089_00000_0000000001_00000?, parameters.CurOffset=89
2025-11-26T14:16:58.628703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 89.0/1
2025-11-26T14:16:58.628707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-26T14:16:58.629259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6817031 count 26 nextOffset 90 batches 13
2025-11-26T14:16:58.629262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 89:0 appended
2025-11-26T14:16:58.629267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-26T14:16:58.629268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.629272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000090_00000_0000000001_00000?, parameters.CurOffset=90
2025-11-26T14:16:58.629346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 90.0/1
2025-11-26T14:16:58.629351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-26T14:16:58.629375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7079245 count 27 nextOffset 91 batches 14
2025-11-26T14:16:58.629377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 90:0 appended
2025-11-26T14:16:58.629382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-26T14:16:58.629383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.629386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000091_00000_0000000001_00000?, parameters.CurOffset=91
2025-11-26T14:16:58.629442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 91.0/1
2025-11-26T14:16:58.629445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-26T14:16:58.629982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7341418 count 28 nextOffset 92 batches 14
2025-11-26T14:16:58.629987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 91:0 appended
2025-11-26T14:16:58.629992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-26T14:16:58.629994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.629997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000092_00000_0000000001_00000?, parameters.CurOffset=92
2025-11-26T14:16:58.630064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 92.0/1
2025-11-26T14:16:58.630069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-26T14:16:58.630082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7603632 count 29 nextOffset 93 batches 15
2025-11-26T14:16:58.630085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 92:0 appended
2025-11-26T14:16:58.630088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-26T14:16:58.630090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.630093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000093_00000_0000000001_00000?, parameters.CurOffset=93
2025-11-26T14:16:58.630143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 93.0/1
2025-11-26T14:16:58.630146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-26T14:16:58.638874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 93 partNo 0 result is x0000000000_00000000000000000062_00000_0000000031_00000 size 8128009
2025-11-26T14:16:58.638902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000062_00000_0000000031_00000 new key d0000000000_00000000000000000062_00000_0000000031_00000 size 8128009 WTime 1764166618638
2025-11-26T14:16:58.638945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 93 partNo 0 FormedBlobsCount 1 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-26T14:16:58.638954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 93:0 appended
2025-11-26T14:16:58.638963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-26T14:16:58.638967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.638972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000094_00000_0000000001_00000?, parameters.CurOffset=94
2025-11-26T14:16:58.639074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 94.0/1
2025-11-26T14:16:58.639079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-26T14:16:58.639397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 524387 count 2 nextOffset 95 batches 1
2025-11-26T14:16:58.639406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 94:0 appended
2025-11-26T14:16:58.639414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-26T14:16:58.639417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-26T14:16:58.639420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000095_00000_0000000001_00000?, parameters.CurOffset=95
2025-11-26T14:16:58.639493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 95.0/1
2025-11-26T14:16:58.639498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-26T14:16:58.639515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 786601 count 3 nextOffset 96 batches 2
2025-11-26T14:16:58.639518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 95:0 appended
2025-11-26T14:16:58.639599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,3 HeadOffset 62 endOffset 64 curOffset 96 d0000000000_00000000000000000093_00000_0000000003_00000| size 786591 WTime 1764166618638
2025-11-26T14:16:58.639748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736488:3709] destroyed
2025-11-26T14:16:58.639759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736492:3713], now have 1 active actors on pipe
2025-11-26T14:16:58.639811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.639818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.639839Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.639853Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 31 size 8128009
2025-11-26T14:16:58.639861Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 3 size 786591
2025-11-26T14:16:58.640401Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-26T14:16:58.640638Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 2 blobs, size 16256018 count 63 last offset 31, current partition end offset: 96
2025-11-26T14:16:58.640643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-26T14:16:58.640660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-26T14:16:57.000000Z
2025-11-26T14:16:58.640664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.640684Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 4. All 2 blobs are from cache.
2025-11-26T14:16:58.640723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-26T14:16:58.641031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.641267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.641660Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.641878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.642433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.643004Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.643544Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.644094Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.644456Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-26T14:16:58.644476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-26T14:16:58.645128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.645690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.646259Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.646664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.647085Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.647668Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.648212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.648623Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-26T14:16:58.648734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-26T14:16:58.649444Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.649946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.650452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.650815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.651160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.651532Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.651917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.652416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.652511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.652515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.652517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.652520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.652522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.652575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.672106Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.679342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 31 size 8128009 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.679361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 3 size 786591 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.679379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.679730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-26T14:16:58.679741Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-26T14:16:58.679746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.679749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 3 parts 0 suffix '124' size 786591
2025-11-26T14:16:58.679841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.679848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.679852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.679854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.679856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.679904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.679916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 64 size 262204 cause it's been evicted from L2. Actual L1 size: 36
2025-11-26T14:16:58.679919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 62 size 524387 cause it's been evicted from L2. Actual L1 size: 35
2025-11-26T14:16:58.679924Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-26T14:16:58.679928Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-26T14:16:58.679937Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000064_00000_0000000001_00000?(+) to d0000000000_00000000000000000064_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679939Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000065_00000_0000000001_00000?(+) to d0000000000_00000000000000000065_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679941Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000066_00000_0000000001_00000?(+) to d0000000000_00000000000000000066_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679944Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000067_00000_0000000001_00000?(+) to d0000000000_00000000000000000067_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679947Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000068_00000_0000000001_00000?(+) to d0000000000_00000000000000000068_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679949Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000069_00000_0000000001_00000?(+) to d0000000000_00000000000000000069_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679951Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000070_00000_0000000001_00000?(+) to d0000000000_00000000000000000070_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679952Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000071_00000_0000000001_00000?(+) to d0000000000_00000000000000000071_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679956Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000072_00000_0000000001_00000?(+) to d0000000000_00000000000000000072_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679958Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000073_00000_0000000001_00000?(+) to d0000000000_00000000000000000073_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679960Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000074_00000_0000000001_00000?(+) to d0000000000_00000000000000000074_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679962Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000075_00000_0000000001_00000?(+) to d0000000000_00000000000000000075_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679965Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000076_00000_0000000001_00000?(+) to d0000000000_00000000000000000076_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679966Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000077_00000_0000000001_00000?(+) to d0000000000_00000000000000000077_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679969Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000078_00000_0000000001_00000?(+) to d0000000000_00000000000000000078_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679971Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000079_00000_0000000001_00000?(+) to d0000000000_00000000000000000079_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679973Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000080_00000_0000000001_00000?(+) to d0000000000_00000000000000000080_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679975Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000081_00000_0000000001_00000?(+) to d0000000000_00000000000000000081_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679977Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000082_00000_0000000001_00000?(+) to d0000000000_00000000000000000082_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679981Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000083_00000_0000000001_00000?(+) to d0000000000_00000000000000000083_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679983Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000084_00000_0000000001_00000?(+) to d0000000000_00000000000000000084_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679985Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000085_00000_0000000001_00000?(+) to d0000000000_00000000000000000085_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679987Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000086_00000_0000000001_00000?(+) to d0000000000_00000000000000000086_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679989Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000087_00000_0000000001_00000?(+) to d0000000000_00000000000000000087_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679992Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000088_00000_0000000001_00000?(+) to d0000000000_00000000000000000088_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679994Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000089_00000_0000000001_00000?(+) to d0000000000_00000000000000000089_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679996Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000090_00000_0000000001_00000?(+) to d0000000000_00000000000000000090_00000_0000000001_00000?(+)
2025-11-26T14:16:58.679998Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000091_00000_0000000001_00000?(+) to d0000000000_00000000000000000091_00000_0000000001_00000?(+)
2025-11-26T14:16:58.680000Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000092_00000_0000000001_00000?(+) to d0000000000_00000000000000000092_00000_0000000001_00000?(+)
2025-11-26T14:16:58.680002Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000093_00000_0000000001_00000?(+) to d0000000000_00000000000000000093_00000_0000000001_00000?(+)
2025-11-26T14:16:58.680004Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000094_00000_0000000001_00000?(+) to d0000000000_00000000000000000094_00000_0000000001_00000?(+)
2025-11-26T14:16:58.680005Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000095_00000_0000000001_00000?(+) to d0000000000_00000000000000000095_00000_0000000001_00000?(+)
2025-11-26T14:16:58.680007Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000002_00000|(+) to d0000000000_00000000000000000062_00000_0000000002_00000|(+)
2025-11-26T14:16:58.681252Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 65 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 66 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 67 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681271Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 68 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 69 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 70 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 71 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 72 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 73 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 74 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 75 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 76 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681312Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 77 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 78 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 79 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 80 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 81 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 82 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681339Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 83 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 84 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681346Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 85 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 86 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 87 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 88 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681361Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 89 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 90 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681369Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 91 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 92 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 93 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681380Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 94 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 95 count 1 actorID [2:7577037922626401473:2365]
2025-11-26T14:16:58.681406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-26T14:16:58.681423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-26T14:16:58.681433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.681435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.681437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.681439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.681441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.681447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-26T14:16:58.681526Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681540Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681555Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681578Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681589Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.681623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-26T14:16:58.693545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736492:3713] destroyed
CallPersQueueGRPC response:
2025-11-26T14:16:58.756384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.756407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.756410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.756414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.756416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
CallPersQueueGRPC request to localhost:28333
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-26T14:16:58.784360Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-26T14:16:58.788339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2718: [PQ: 72075186224037892] server connected, pipe [1:7577037931283736503:3721], now have 1 active actors on pipe
2025-11-26T14:16:58.788406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-26T14:16:58.788415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2632: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-26T14:16:58.788472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-26T14:16:58.788525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 16256018 count 62 last offset 31, current partition end offset: 96
2025-11-26T14:16:58.788531Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-26T14:16:58.788553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.788559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-26T14:16:58.000000Z
2025-11-26T14:16:58.788584Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-26T14:16:58.788633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-26T14:16:58.789593Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.790141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.790644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.791104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.791683Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.791908Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.792127Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.792321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-26T14:16:58.792327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-26T14:16:58.796862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.797160Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.797419Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.797672Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.797916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.798162Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.798381Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.798609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.798754Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-26T14:16:58.798887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-26T14:16:58.799095Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.799330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.799579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.799818Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.808575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.808913Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.809182Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.809416Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-26T14:16:58.809549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-26T14:16:58.842044Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-26T14:16:58.859825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.859845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.859848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.859851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.859854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:58.876331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2743: [PQ: 72075186224037892] server disconnected, pipe [1:7577037931283736503:3721] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-26T14:16:58.957632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:58.957648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.957651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:58.957654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:58.957657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:59.059877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:59.059906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.059909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:59.059913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.059916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:59.164445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:59.164470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.164473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:59.164478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.164481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-26T14:16:59.268358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-26T14:16:59.268385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.268388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-26T14:16:59.268393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-26T14:16:59.268395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3344, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext &): (info_half1.BlobsFromDisk == 1) failed: (0 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x150A84EC)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x152644F0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext&)+5702 (0x14DDE496)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14EBF347)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x152666C8)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14EBEC44)
NUnitTest::TTestFactory::Execute()+817 (0x15266E11)
NUnitTest::RunMain(int, char**)+3229 (0x1527927D)
??+0 (0x7F65D67C7D90)
__libc_start_main+128 (0x7F65D67C7E40)
_start+41 (0x13EC7029)
[[rst]]
