Let the source location's data processor copy pulled data to the location it targets

[SylvainSenechal]

BB-913

With pull replication, the sink copies each object's data from a remote source location (for example dr-source) into a local location (for example us-east-1). The copy is a copyLocation action with fromLocation = the source and toLocation = the local target.

Two problems in the data processor:

The copy was never performed. Backbeat has one data processor per replication site (a remote location such as dr-source, or a cloud location), and each data processor only performs the copies whose target is its own site. Here the target is the local location (us-east-1). A local location is not a replication site, so it has no data processor, and no data processor performed the copy. The data processor that should perform it is the one of the source location (dr-source): it's the only one with that location's address and credentials, so the only one able to read the data.

Once given to the data processor of dr-source, the copy would go to the wrong place. A data processor always writes the copied data to its own site, because until now its own site was always the target of the copy. With pull replication that's no longer true: the data processor of dr-source performs the copy, but the target is us-east-1. Writing "to its own site" would send the data back to dr-source instead of us-east-1. Cloudserver also needs the storage type of the location it writes to, and that type wasn't sent. So the data processor must write to the target of the copy (us-east-1), with that location's type.

[bert-e]

Hello sylvainsenechal,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/bypass_source_branch_lineage	Bypass the cross-branch contamination check	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request.
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[codecov]

Codecov Report

❌ Patch coverage is 86.04651% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.41%. Comparing base (64dd89f) to head (2f1f046).

Files with missing lines	Patch %	Lines
extensions/replication/tasks/CopyLocationTask.js	81.25%	6 Missing ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
extensions/replication/ReplicationAPI.js	91.77% <100.00%> (+0.05%)	⬆️
...sions/replication/queueProcessor/QueueProcessor.js	79.64% <100.00%> (+0.57%)	⬆️
extensions/replication/tasks/CopyLocationTask.js	76.74% <81.25%> (+0.27%)	⬆️

... and 3 files with indirect coverage changes

Components	Coverage Δ
Bucket Notification	85.50% <ø> (ø)
Core Library	87.24% <ø> (+0.36%)	⬆️
Ingestion	83.38% <ø> (ø)
Lifecycle	86.24% <ø> (ø)
Oplog Populator	89.48% <ø> (ø)
Replication	68.79% <86.04%> (+0.14%)	⬆️
Bucket Scanner	89.37% <ø> (ø)

@@                 Coverage Diff                 @@
##           development/9.6    #2876      +/-   ##
===================================================
+ Coverage            81.24%   81.41%   +0.17%     
===================================================
  Files                  209      209              
  Lines                44290    44319      +29     
===================================================
+ Hits                 35985    36084      +99     
+ Misses                8305     8235      -70     

Flag	Coverage Δ
api:retry	14.56% <0.00%> (-0.01%)	⬇️
api:routes	14.58% <0.00%> (-0.01%)	⬇️
bucket-scanner	89.37% <ø> (ø)
ft_test:queuepopulator	18.60% <4.65%> (+1.24%)	⬆️
ingestion	20.53% <4.65%> (-0.02%)	⬇️
lib	12.89% <0.00%> (-0.01%)	⬇️
lifecycle	26.77% <6.97%> (-0.02%)	⬇️
notification	1.89% <0.00%> (-0.01%)	⬇️
oplogPopulator	0.45% <0.00%> (-0.01%)	⬇️
replication	25.29% <79.06%> (+0.05%)	⬆️
unit	67.51% <69.76%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[SylvainSenechal]

This is duplicated data with line 56 so its extra duplicated data only for the sake of clarity but 🤷‍♂️

Tell me if you would keep it or nah