summaryrefslogtreecommitdiffstats
path: root/sync/engine
Commit message (Collapse)AuthorAgeFilesLines
* [Sync] Re-enable experiments datatypezea@chromium.org2012-11-171-3/+3
| | | | | | | | | | | It was disabled while a server bug was fixed. The fix has now rolled out. BUG=None Review URL: https://chromiumcodereview.appspot.com/11412057 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@168431 0039d316-1c4b-4281-b951-d872f2087c98
* Added directory entry counters to about:syncvishwath@google.com2012-11-151-0/+4
| | | | | | | | | | | | | | Added counters that show the following information on a per ModelType basis - Total Entries Live Entries The information is propagated via TakeSnapshot(). BUG=151669 Review URL: https://chromiumcodereview.appspot.com/10993030 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@168001 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Update calls from RunAllPending() to RunUntilIdle().tfarina@chromium.org2012-11-141-3/+3
| | | | | | | | | | | | RunAllPending() is deprecated and we should switch to RunUntilIdle(). BUG=131220 TBR=akalin@chromium.org Review URL: https://chromiumcodereview.appspot.com/11275305 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@167651 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Disable experiments datatype temporarilyzea@chromium.org2012-11-141-3/+3
| | | | | | | | | | | A server bug currently means enabling this datatype results in unrecoverable errors. Disable it temporarily. BUG=None Review URL: https://codereview.chromium.org/11366231 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@167530 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Add application logic for non-nigori control types (reland)zea@chromium.org2012-11-135-45/+196
| | | | | | | | | | | | Relanding after revert. Original codereview at https://chromiumcodereview.appspot.com/11271009/. BUG=122825 TBR=rlarcoque@chromium.org, akalin@chromium.org Review URL: https://codereview.chromium.org/11361242 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@167466 0039d316-1c4b-4281-b951-d872f2087c98
* Revert 167280 - [Sync] Add application logic for non-nigori control types.zea@chromium.org2012-11-135-195/+45
| | | | | | | | | | | | | | | | | | | | | We introduce logic to apply non-nigori control types. First, we iterate over any new top level datatype entities, applying those. Then we go through the rest of the unapplied control datatype updates, applying those. Any conflict should be just a simple conflict, which we handle by ignoring the local changes. Also updates chromiumsync.py to support the new control types (and fixes the parent folder pattern that was in use). BUG=122825 Review URL: https://chromiumcodereview.appspot.com/11271009 TBR=zea@chromium.org Review URL: https://codereview.chromium.org/11359175 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@167282 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Add application logic for non-nigori control types.zea@chromium.org2012-11-135-45/+195
| | | | | | | | | | | | | | | | | | We introduce logic to apply non-nigori control types. First, we iterate over any new top level datatype entities, applying those. Then we go through the rest of the unapplied control datatype updates, applying those. Any conflict should be just a simple conflict, which we handle by ignoring the local changes. Also updates chromiumsync.py to support the new control types (and fixes the parent folder pattern that was in use). BUG=122825 Review URL: https://chromiumcodereview.appspot.com/11271009 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@167280 0039d316-1c4b-4281-b951-d872f2087c98
* sync: fix for bug 158313, which is a top crash on canary.tim@chromium.org2012-11-063-0/+43
| | | | | | | | | | | | | | | | If a nudge is scheduled after a connection change canary, an abandoned job may wind up in DoCanaryJob. Since the job knows by design if it's in this state, this patch adds a defensive check in DoCanaryJob to check (similar to the one in DoSyncSessionJob). We should be able to remove the CreateSyncSession code that actually tries to use the job before calling DoSyncSessionJob, though. BUG=158313 Review URL: https://chromiumcodereview.appspot.com/11366079 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@166215 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Follow-up to conflict resolution refactorrlarocque@chromium.org2012-11-019-40/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is part two of r164286. That commit refactored the way we handle conflict resolution. This commit takes advantage of those changes to delete lots of code. Because this change deletes session_state.cc, I decided to move the two remaining useful tests session_state_unittest.cc into their own files, sync_session_snapshot_unittest.cc and sync_source_info_unittest.cc. The tests were not modified in any way. None of these changes should have any effect on syncer behaviour: - We can remove the simple conflict counters and related code, since we now assert that all conflicts will be resolved by the end of a successful sync cycle. - We can remove the PerModelSafeGroupState because that struct no longer has any members. - The 'conflicts_resolved' indicators are no longer set, so we can remove them. - Without those indicators, it's no longer possible to have any SYNC_CYCLE_CONTINUATION sync cycles. We can remove a few counters associated with them, as well as the 'has_more_to_sync' flag in the snapshot. - Without SYNC_CYCLE_CONTINUATION cycles, there's no longer any need for code that loops around SyncShare() in SyncSchedulerImpl. The SyncSession::PrepareForAnotherSyncCycle() function is no longer used, either. - The ScopedConflictResolver installed on the session during a sync cycle is no longer used, so all the code related to it can be deleted. BUG=147681,111280,156238 Review URL: https://chromiumcodereview.appspot.com/11314008 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@165474 0039d316-1c4b-4281-b951-d872f2087c98
* sync: add kInitialBackoffImmediateRetry tim@chromium.org2012-11-013-10/+10
| | | | | | | | | | | | | | | | | | | | | | The primary impact this has is speeding up some integration tests by ~ 50%. In particular it lets us re-enable MigrationHell, which is currently disabled as it takes too long for sharding_supervisor. Rationale is discussed in the bug, I believe this is safe for the class of errors we target with the immediate delay, because we don't need backoff. (Note that when used with GetDelay, this still results in a 1s minimum backoff.) kInitialBackoffShortRetry is still used if the command line flag is passed (and in integration tests) to change the default initial backoff from 5 minutes to 1s (as it used to be). BUG=143641 Review URL: https://chromiumcodereview.appspot.com/11183073 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@165312 0039d316-1c4b-4281-b951-d872f2087c98
* sync: reland scheduler ownership refactoring, now with less crashtim@chromium.org2012-10-3011-477/+1072
| | | | | | | | | | | | | Prevent abandoned jobs from entering DoCanaryJob by stopping the timer in TakePendingJobForCurrentMode. Original review at https://codereview.chromium.org/10917234/ BUG=158313 Review URL: https://chromiumcodereview.appspot.com/11341030 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@164992 0039d316-1c4b-4281-b951-d872f2087c98
* Revert 164565 - sync: make scheduling logic and job ownership more obvious.dharani@google.com2012-10-2911-1038/+477
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Revert Reason - see bug 158313 - inlines (and removes) SaveJob, to perform only the relevant "saving" bits where needed. As it turns out, most of it was only necessary for ShouldRunJob (which I'd like to rename, as it's destructive), and it's a lot easier to read what's happening and why. Note also removed TODO at SaveJob callsites. - inlines (and removes) InitOrCoalescePendingJob to perform only the relevant bits at each callsite. - pulls SyncSessionJob into a class, to handle basic responsibilities that need the job Purpose + session (like "Succeeded()" and "Finish") that were previously strewn about and partially copy/pasted in the scheduler. - meticulously transfers ownership (removes linked_ptr usage!) of jobs instead of making many implicit struct copies, as it resulted in non-obvious behavior. To do this, ownership is given to the scheduling mechanism (the MessageLoop for ScheduleSyncSessionJob, WaitInterval::timer for canary jobs), instead of jobs sitting around without knowing whether they're scheduled or not. There are a few cases where we can't actually "schedule" a job -- which wasn't even obvious from the old code, and other interesting revelations, like fact that we were actually pre-empt nudge canary jobs and "unscheduling" them in certain cases. - removes DoPendingJobIfPossible, since it is no longer needed for DoCanaryJob/Unthrottle cases, and make the behavior more explicit in the other cases (mode-switch and SCM error change), see TakePendingJobForCurrentMode. - removes the ability to create jobs as canary, since this wasn't needed and was adding complexity (see DoCanaryJob / GrantCanaryPrivilege). - removes retry_scheduled as it wasn't used anywhere - adds lots of comments. Also discovered that THROTTLED intervals seem to never fire a canary job with today's code (broken), config jobs are immune to per-data-type throttling, and the had_nudge allowance was defeated by extra IsBackingOff check in ScheduleNudgeImpl (see comment on review). BUG= Review URL: https://chromiumcodereview.appspot.com/10917234 TBR=tim@chromium.org Review URL: https://codereview.chromium.org/11347027 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@164780 0039d316-1c4b-4281-b951-d872f2087c98
* sync: make scheduling logic and job ownership more obvious. tim@chromium.org2012-10-2811-477/+1038
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - inlines (and removes) SaveJob, to perform only the relevant "saving" bits where needed. As it turns out, most of it was only necessary for ShouldRunJob (which I'd like to rename, as it's destructive), and it's a lot easier to read what's happening and why. Note also removed TODO at SaveJob callsites. - inlines (and removes) InitOrCoalescePendingJob to perform only the relevant bits at each callsite. - pulls SyncSessionJob into a class, to handle basic responsibilities that need the job Purpose + session (like "Succeeded()" and "Finish") that were previously strewn about and partially copy/pasted in the scheduler. - meticulously transfers ownership (removes linked_ptr usage!) of jobs instead of making many implicit struct copies, as it resulted in non-obvious behavior. To do this, ownership is given to the scheduling mechanism (the MessageLoop for ScheduleSyncSessionJob, WaitInterval::timer for canary jobs), instead of jobs sitting around without knowing whether they're scheduled or not. There are a few cases where we can't actually "schedule" a job -- which wasn't even obvious from the old code, and other interesting revelations, like fact that we were actually pre-empt nudge canary jobs and "unscheduling" them in certain cases. - removes DoPendingJobIfPossible, since it is no longer needed for DoCanaryJob/Unthrottle cases, and make the behavior more explicit in the other cases (mode-switch and SCM error change), see TakePendingJobForCurrentMode. - removes the ability to create jobs as canary, since this wasn't needed and was adding complexity (see DoCanaryJob / GrantCanaryPrivilege). - removes retry_scheduled as it wasn't used anywhere - adds lots of comments. Also discovered that THROTTLED intervals seem to never fire a canary job with today's code (broken), config jobs are immune to per-data-type throttling, and the had_nudge allowance was defeated by extra IsBackingOff check in ScheduleNudgeImpl (see comment on review). BUG= Review URL: https://chromiumcodereview.appspot.com/10917234 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@164565 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Merge apply updates and resolve conflictsrlarocque@chromium.org2012-10-2615-399/+282
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The conflict resolution code was executed after the commit for reasons which no longer apply. Because we no longer have to worry about resolving hierarchy conflicts or nigori node conflicts, we have the opportunity to move conflict resolution closer to update application. One advantage of resolving conflicts early is that we no longer require an extra sync cycle to commit any 'local wins' conflict resolutions. This makes SYNC_CYCLE_CONTINUATION sync cycles obsolete. Another advantage is that update application and conflict resolution can be performed without releasing the sync lock, which eliminates several types of races that we used to have to worry about. It's probably more efficient, too. It allows us to guarantee that there are no simple conflicts remaining after the update application step is completed. The effects might not be very noticeable to end users, but it will be nice to remove some of the conflict tracking code. Finally, it removes the last use PerModelSafeGroupState. This will let us delete some unused code and hopefully simplify StatusController. This patch does not pursue these cleanups as agressively as it could. The idea here is to keep the complex logic changes in one small CL, and perform the cleanups in a larger, but simpler, follow-up CL. BUG=147681,11280,156238 Review URL: https://chromiumcodereview.appspot.com/11192071 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@164286 0039d316-1c4b-4281-b951-d872f2087c98
* sync: wrap conflict_utils in a namespacetim@chromium.org2012-10-254-6/+12
| | | | | | | | | | | This makes it easier to hunt down function definitions and matches file hierarchy as suggested by the style guide. BUG= Review URL: https://chromiumcodereview.appspot.com/11238043 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@163992 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Fix undelete conflict resolutionzea@chromium.org2012-10-232-30/+15
| | | | | | | | | | | | | | | | | | Previously we were recreating bookmarks when undeleting, and resetting the version of non-bookmarks. Neither of these are necessary. We now reuse the current version (so the server knows we are the most up to date versin) and no longer re-creae bookmarks (because we reuse the existing id, the write is treated as a modify, not a create). BUG=154884 TEST=with three signed in clients A, B, C, delete a bookmark on A. On B and C, before they recieve the notification, modify that same bookmark. Ensure no duplicates are created. Review URL: https://chromiumcodereview.appspot.com/11204002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@163451 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Remove UpdateProgress and related coderlarocque@chromium.org2012-10-175-211/+57
| | | | | | | | | | | | | | | | | | Following the merge of the process updates and verify updates step, there is no longer any need for the UpdateProgress class. Its only use outside of tests was to pass data between the verify and update steps during a sync cycle. The class had some use in tests, but most of the assertions based on it were somewhat redundant. They can be replaced by assertions on similar, but non-ModelSafe, counts. BUG=156238 Review URL: https://chromiumcodereview.appspot.com/11190013 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@162576 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Merge {Verify,Process}UpdatesCommandrlarocque@chromium.org2012-10-168-412/+251
| | | | | | | | | | | | | | | | | | | | The VerifyUpdatesCommand was used to ensure the validity of each update that had been delivered from the server and to prepare for the ProcessUpdatesCommand. The ProcessUpdatesCommand would then take all updates that had been successfully verified and move them into the SERVER_ fields of the sync directory. It turns out that the logic can be greatly simplified by performing both tasks within the same command. This patch does not take full advantage of the merge. This patch is intended simply to merge the two files, with the goal of performing more significant refactorings later. BUG=154654 Review URL: https://chromiumcodereview.appspot.com/11091009 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@162176 0039d316-1c4b-4281-b951-d872f2087c98
* Extend sync protocol to report hierarchy conflictsrlarocque@chromium.org2012-10-163-0/+43
| | | | | | | | | | | | | | | | | | | | | | This commit adds a new submessage to the protocol buffer that can be used to report the client's status to the server. The first status indicator to be implemented is 'hierarchy_conflict_detected', which is an implicit call for help. In the future, the server should respond to this flag by tring to resolve the hierarchy conflicts. To help manually test the new field, this change also modified the proto to DictionaryValue conversion functions to no longer print unset fields. This is important because the client must leave the 'hierarchy_conflict_detected' field unset until it has tried to apply updates at least once. This also had some implications for the unit tests. BUG=152464 Review URL: https://chromiumcodereview.appspot.com/11090052 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@162053 0039d316-1c4b-4281-b951-d872f2087c98
* sync: return short initial backoff delay for NETWORK_CONNECTION_UNAVAILABLEtim@chromium.org2012-10-122-0/+32
| | | | | | | | | BUG=chromium-os:35073 Review URL: https://chromiumcodereview.appspot.com/11098084 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@161481 0039d316-1c4b-4281-b951-d872f2087c98
* Cleanup: IWYU for base/time.h in sync/.thestig@chromium.org2012-10-103-1/+6
| | | | | | Review URL: https://chromiumcodereview.appspot.com/11028086 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@161059 0039d316-1c4b-4281-b951-d872f2087c98
* Changed DB to store node positions as Ordinals.vishwath@google.com2012-10-096-17/+50
| | | | | | | | | | | | | | As part of the effort to move away from int64 based node positions, changed the DB to store server_position_in_node as an ordinal (represented by a varchar) instead. Also updated unittests and the latest db version number (now at 81). BUG=145412 Review URL: https://chromiumcodereview.appspot.com/10989063 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@160774 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Remove ConflictProgressrlarocque@chromium.org2012-10-069-166/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | This change removes the ConflictProgress struct from the PerModelSafeGroupState. The struct was intended to pass state between the update application and conflict resolution commands. This part of its functionality has been replaced with a vector of simple conflict item IDs. The ConflictProgress struct also had an important role to play in stats gathering and unit tests. To replace it, the update applicator has been updated to store its counts directly in the StatusController. Unlike ConflictProgress, this state is not thread-local. That's acceptable because we can guarantee it will only be read or written by one thread at a time and those threads use appropriate thread-safety mechanisms to transfer control to each other. This change is another step towards merging update application and conflict resolution. BUG=147681 Review URL: https://chromiumcodereview.appspot.com/11049002 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@160530 0039d316-1c4b-4281-b951-d872f2087c98
* Rename ModelType/ObjectIdStateMap to InvalidationMap.dcheng@chromium.org2012-10-047-61/+63
| | | | | | | | | | BUG=none TBR=sky Review URL: https://chromiumcodereview.appspot.com/11052007 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@160073 0039d316-1c4b-4281-b951-d872f2087c98
* Create the mobile bookmarks folder both on Android and iOS.msarda@chromium.org2012-10-031-1/+1
| | | | | | Review URL: https://chromiumcodereview.appspot.com/10963053 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@159880 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Don't use ConflictProgress in the resolverrlarocque@chromium.org2012-10-025-36/+20
| | | | | | | | | | | | | | | | | | | | Remove all references to the ConflictProgress struct from the place where it is needed most: the ConflictResolver. It turns out that the ConflictResolver's only relevant input is a set of simple conflict IDs, and it has no need for the rest of the ConflictProgress struct. This change is a step towards the removal of the ConflictProgress struct, which will eventually allow us to improve the way that the syncer handles conflicts. This commit also cleans up some includes and forward declarations in the conflict resolver. BUG=147681 Review URL: https://chromiumcodereview.appspot.com/10989087 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@159621 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Refactor update applicationrlarocque@chromium.org2012-09-285-115/+87
| | | | | | | | | | | | | | | | This is a refactoring patch. It removes an unnecessary ConflictResolver parameter from a few helper functions and rewrites the update application logic. This patch is part of a series. I expect that future patches will be easier to implement if an applicator can be reused, and the iterator members make it difficult to do so with the current design. BUG=147681 Review URL: https://chromiumcodereview.appspot.com/10964057 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@159186 0039d316-1c4b-4281-b951-d872f2087c98
* Added nudge source counters to about:sync.vishwath@google.com2012-09-252-0/+26
| | | | | | | | | | | | | Added counters for each possible nudge source on the summary tab of the about:sync page. The counters are stored in SyncStatus and managed by SyncManagerImpl via allstatus_. BUG=78166 Review URL: https://chromiumcodereview.appspot.com/10991005 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@158633 0039d316-1c4b-4281-b951-d872f2087c98
* Removed safe worker calculation from SyncScheduler.vishwath@google.com2012-09-201-22/+5
| | | | | | | | | | | | | | Removed the calculation of restricted_workers from BuildModelSafeParams and updated the SyncSession constructor to no longer require a workers parameter. These changes are in line with bug 133030 and the change to model changing commands so that they no longer post to unused workers. BUG=133030 Review URL: https://chromiumcodereview.appspot.com/10947039 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@157852 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Add keystore encryption info to about:synczea@chromium.org2012-09-192-0/+18
| | | | | | | | | | | | | | | | | | | This patch adds the following fields under the encryption section of about:sync - Has Keystore Key: whether the encryption handler has a keystore encryption key - Migration Time: the time migration was performed, or "Not Migrated" if migration hasn't been performed yet - Passphrase Type: the actual passphrase type (provides more detail than Is Using Explicit Passphrase, but stored at a diff layer) Added sync/api/time.h, which just includes sync/util/time.h but is accessible from chrome/ BUG=129665 Review URL: https://chromiumcodereview.appspot.com/10917246 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@157499 0039d316-1c4b-4281-b951-d872f2087c98
* sync: add comment to SyncerProtoUtiltim@chromium.org2012-09-171-0/+2
| | | | | | | | | | Follow up from http://codereview.chromium.org/10916276/ BUG=none Review URL: https://chromiumcodereview.appspot.com/10911341 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@157198 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Add keystore migration conflict support.zea@chromium.org2012-09-173-210/+634
| | | | | | | | | | | | | We rely on the encryption handler's rollback detection for any cases where conflicts might lose data and we can't merge. Else, we merge the data as needed. BUG=129665 Review URL: https://chromiumcodereview.appspot.com/10905191 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@157186 0039d316-1c4b-4281-b951-d872f2087c98
* sync: add Google API Key to all ClientToServerMessages.tim@chromium.org2012-09-175-22/+17
| | | | | | | | | | | | | Also move setting of "must have" sync fields to PostClientToServerMessage (from DownloadUpdatesCommand & BuildCommitCommand). Needed for M23 branch. BUG= Review URL: https://chromiumcodereview.appspot.com/10916276 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@157081 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Implement keystore migration support.zea@chromium.org2012-09-133-17/+18
| | | | | | | | | | | | | | | | | | | | | | | We'll now trigger migration if the keystore key is available, the cryptographer is ready, and the nigori node isn't already properly migrated. Note that this means we won't trigger migration without at least the implicit gaia password already available to the cryptographer, in order to support backwards compatibility with older clients. Eventually that will change. In addition, once a nigori node has been migrated, any client that supports keystore encryption will follow the new encryption constraints, whether or not the --sync-keystore-encryption flag is passed. This means that if the user sets a custom passphrase, encrypt everything will also be enabled (and vice versa). Migration-aware conflict resolution is not implemented yet. BUG=129665 Review URL: https://chromiumcodereview.appspot.com/10916036 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@156646 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Process client commands on commit responserlarocque@chromium.org2012-09-124-40/+64
| | | | | | | | | | | | | | The server sends down client commands on both commit and get update responses. Prior to this change, the client would only react to the commands sent in the get update response. Now it will respond to commands sent in either message. BUG=147680 Review URL: https://chromiumcodereview.appspot.com/10909160 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@156424 0039d316-1c4b-4281-b951-d872f2087c98
* Implement a bag of chips for sync.qsr@chromium.org2012-09-114-0/+34
| | | | | | | | | | | | | The bag of chips is a per-client state used by the server. This state is stored on the client and send back to the server on all communication. BUG=None TEST=None Review URL: https://chromiumcodereview.appspot.com/10916174 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@156123 0039d316-1c4b-4281-b951-d872f2087c98
* sync: remove buggy freshness condition in SyncSchedulerImpl add defensive ↵tim@chromium.org2012-09-074-59/+143
| | | | | | | | | | | | | | | | | | | | | | | DCHECKs. We were using last_sync_session_end_time, which would effectively allow config jobs to cancel nudges that were scheduled to start before the config ended. There was code to reset the start time of nudges to hack around this, (which is removed in this patch) which was pretty confusing because after executing if (scheduled_start < Now()) scheduled_start = Now(); scheduled_start is less than Now() again. This check was purely to get around the freshness check (AFAICT), and I think the new code is clearer. It also affects the Sync.Freq histogram, see bug for detail. The integration tests also caught the fact that we weren't checking Succeeded() before updating that time, which means failed jobs could have caused legitimate nudges to be cancelled -- note this probably wasn't a big problem in practice due to the hack mentioned above. We now check Succeeded(). This also adds a few DCHECKs for situations that were previously unsupported but would technically squeak through. For example, there's a DCHECK in ScheduleConfiguration that there is no pending config job, but that situation could have arisen if we swapped to NORMAL_MODE and back before the config job executed -- in fact, one of the tests was doing this, but it's clearly unsupported. Also fixes an issue with test_util::SimulateSuccess where we would not set ModelNeutralState results for config jobs, which meant that in some cases, tests would *not* reset wait_interval_ on successful backoff relief. The changes to the test file now ensure this can't regress. BUG=143595 Review URL: https://chromiumcodereview.appspot.com/10825439 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@155370 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Pass ModelTypeSet by value everywhereakalin@chromium.org2012-09-017-8/+8
| | | | | | | | | | | ModelTypeSet is small enough to not need to be passed around by const reference. BUG=145986 Review URL: https://chromiumcodereview.appspot.com/10905045 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@154595 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Introduce control data typesrlarocque@chromium.org2012-08-3111-494/+623
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Nigori node has always been treated specially. It is downloaded during early sync initialization. It has custom encryption, update application, and conflict resolution logic. It is not user-selectable, but is instead always implicitly enabled. This patch replaces a lot of if (NIGORI)' code branches and creates the infrastructure we will need to support other data types that have these properties. These types will be used to store metadata required for the normal operation of the syncer. Notable changes include: - Introduce mutually exclusive lists of control types and user types in model_type.cc. - Create an ApplyControlDataUpdates() function, which will be used to apply changes and resolve conflicts for control data types. - Add some if statements to skip processing of control data updates in the regular conflict resolution code path. - Move around some code to accomodate the above changes. Introduce apply_control_data_updates_unittest.cc and conflict_util.cc - Have SyncBackendHost download all control types during init, not just the NIGORI. - Introduce a PurgeForMigration() function on the DataTypeManagerImpl class to support migration of control data types. This replaces ConfigureWithoutNigori(), which would not have scaled well as additional control data types were addded. - Modify the encryption logic so that only user types may be set to be encrypted. The Nigori is no longer expected to be in the set of encrypted types, though it is no less encrypted than it was before this change. - Use the list of user types to help calculate the list of registered sync prefs. BUG=122825,111297 Review URL: https://chromiumcodereview.appspot.com/10832286 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@154522 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Move keystore key handling to SyncEncryptionHandlerImplzea@chromium.org2012-08-292-12/+12
| | | | | | | | | | | | The cryptographer has no notion of keystore keys, and we now persist the keystore key by reusing the OnBoostrapTokenUpdated method (which now takes an enum as an extra param specifying the type of token). BUG=129665 Review URL: https://chromiumcodereview.appspot.com/10878015 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@154007 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Refactor GetEncryptedTypes usage.zea@chromium.org2012-08-246-43/+53
| | | | | | | | | | | | | | | | | | | | | | | | The cryptographer is now owned by the SyncEncryptionHandlerImpl, and no longer needs any connection to the nigori handler or encrypted types. Those are accessed via the directory, which now has a GetNigoriHandler method. Because of this, we can now clean up the thread safety guarantees in the sync encryption handler impl, enforcing that everything except GetEncryptedTypes (and IsUsingExplicitPassphrase, which will be fixed in a followup patch) are invoked on the syncer thread. This lets us not have to open unnecessary transactions. At the chrome layer, encrypted types are accessed via BaseTransaction:: GetEncryptedTypes(). At the syncer layer, they're accessed via directory:: GetNigoriHandler()->GetEncryptedTypes(BaseTransaction* trans const). BUG=142476,139848 Review URL: https://chromiumcodereview.appspot.com/10844005 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@153335 0039d316-1c4b-4281-b951-d872f2087c98
* Refactor ModelTypePayloadMap and ObjectIdPayloadMap to StateMaps.dcheng@chromium.org2012-08-248-84/+85
| | | | | | | | | | | | | This lays the groundwork for allowing us to pass ack handles for object IDs to SyncNotifierObservers. BUG=124149 TEST=none, there should be no behavior change. Review URL: https://chromiumcodereview.appspot.com/10837214 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@153158 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Add SyncEncryptionHandlerzea@chromium.org2012-08-164-132/+59
| | | | | | | | | | | | | | | | | | All sync-specific encryption state (types, encrypt everything, explicit passphrase, keys) is now tracked within the new class SyncEncryptionHandler. It's owned by the sync manager, and unifies some of the observer logic we previously had. In addition, it's capable of creating its own transactions, taking us a step closer to have a nigori datatype. In addition, we add a NigoriHandler to abstract the chrome-side of encryption from the sync visible side of encryption. BUG=139848 Review URL: https://chromiumcodereview.appspot.com/10827266 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@151833 0039d316-1c4b-4281-b951-d872f2087c98
* sync: add InternalComponentsFactory::Switches to simplify passing switches ↵tim@chromium.org2012-08-157-160/+265
| | | | | | | | | | | | | | | | | | to internal components. Cleans up backoff retry override code to use InternalComponentsFactory::Switches rather than global bool hack. Also puts keystore encryption flag atop new mechanism. (TBR sky for new chrome_switch). TBR=sky@chromium.org BUG=142029, 139839 Review URL: https://chromiumcodereview.appspot.com/10837231 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@151664 0039d316-1c4b-4281-b951-d872f2087c98
* sync: raise initial backoff interval from 1 second to 5 minutes.tim@chromium.org2012-08-133-2/+90
| | | | | | | | | | | | | | | | | | | Long term, we'd like to drop this further (i.e. 1 minute) and trust THROTTLED responses in the event of widespread backend outages. We'd like to run some stress tests of that infrastructure (at scale / under load), but in the meantime we want to address the problem for beta channel users / M22. We chose 5 minutes, which is a little more aggressive than the recommended value, but we believe is safe -- happy to discuss offline. This patch is very low risk (stability wise). although 5 minutes is a long time to wait for users who hit this due to transient failures. Current data suggests 500s occur around 0.02% of the time, and although backoff isn't limited to 500s it's unlikely to get hit frequently, and the user experience doesn't break down significantly. Effectiveness wise, We can run an experiment once it hits dev channel by returning 500s and watching for expected drops in QPS. BUG=139733,142029 Review URL: https://chromiumcodereview.appspot.com/10826194 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@151216 0039d316-1c4b-4281-b951-d872f2087c98
* sync: Improve tracking of useless updatesrlarocque@chromium.org2012-08-011-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our counts of non-useless updates are wrong in part because of the way we track deletions. When an item is deleted, we flip the IS_DEL bit. On restart, we purge all fully synced IS_DEL items from our directory. However, we might still receive some re-deliveries of the tombstone for that item. The old algorithm would consider tombstones of non-existent items to be "useful", despite the fact that they're often just reflections of updates we've already received. The updated algorithm isn't perfect. There's no easy way to distinguish between a tombstone created by another client where this client never saw the original (which one could argue is useful, and definitely not a reflection) and the re-delivery of a tombstone created by another client which this client has already received and processed long ago (which is definitely not useful). Both scenarios look the same, and both will be counted as 'reflections' under the new algorithm. A similar issue arises with UNIQUE_CLIENT_TAG items. When we delete them we force their version to 0 locally. This makes our regular reflection detection algorithm ineffective. This change updates the algorithm to consider tombstones to items that have UNIQUE_CLIENT_TAG and are locally deleted to be reflections. It's not perfect, but it should be good enough for our purposes. BUG=139684 Review URL: https://chromiumcodereview.appspot.com/10834095 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@149368 0039d316-1c4b-4281-b951-d872f2087c98
* [Sync] Add support for performing a GetKey on startup.zea@chromium.org2012-07-316-28/+93
| | | | | | | | | | | | | | | | | The functionality is behind the --sync-keystore-encryption flag, and the key is not currently consumed by anything, but this lays the groundwork for testing the server and client interaction. We request a key anytime we perform a GetUpdates while the cryptographer does not have a keystore key. But, it is considered an error to request a key and not receive one, putting us into a state of backoff. BUG=129665 TEST=sync_unit_tests, running against python server Review URL: https://chromiumcodereview.appspot.com/10455012 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@149248 0039d316-1c4b-4281-b951-d872f2087c98
* Reland 148792zea@chromium.org2012-07-308-308/+6
| | | | | | | | | | | | | | | | Relanding after rebasing. [Sync] Remove CleanupDisabledTypes command and move purge logic into SyncManager." Original codereview at http://codereview.chromium.org/10541079/ BUG=131433, 90868 TBR=tim@chromium.org Review URL: https://chromiumcodereview.appspot.com/10829086 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@149052 0039d316-1c4b-4281-b951-d872f2087c98
* Make SyncBackendRegistrar aware of loaded datarlarocque@chromium.org2012-07-303-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds a parameter to the SyncManager::Initialize() callback to inform the caller which sync data types were successfully loaded from disk. This allows the SyncBackendHost and related classes make better decisions during initialization. If necessary, the SyncBackendHost will send a configure request to the syncer during early initialization asking it to download the nigori node. Now we can guarantee that the node will be available by the type ProfileSyncService::OnBackendInitialized() is called. The SyncBackendHost test expectations had to be amended to account for this test. Most of the changes are related to the fact that our behaviour no longer depends on the SyncPrefs. The ProfileSyncService*Tests were very much affected by this change. Those tests are parameterized with a callback that is used to initialize the sync DB and pretend that it had been loaded from disk. Previously, this callback would be executed later on during initialization, around the time of ProfileSyncService::OnBackendInitialize(). That approach was incompatible with this change, which requires that we have access to the fully initialized database around the time we return from SyncBackendHost::Initialize(). The callback had to be moved, which meant that it now required an explicit UserShare parameter, which meant that a lot of call sites had to be updated. BUG=129825 TEST= Review URL: https://chromiumcodereview.appspot.com/10804039 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@148926 0039d316-1c4b-4281-b951-d872f2087c98
* Revert 148792 - [Sync] Remove CleanupDisabledTypes command and move purge ↵sergeyu@chromium.org2012-07-278-6/+308
| | | | | | | | | | | | | | | | | | logic into SyncManager. We were only ever performing a meaningful cleanup on reconfigurations or restart, so we make that explicit by purging from within the SyncManager's loading and configuration methods. BUG=131433, 90868 TEST=manual Review URL: https://chromiumcodereview.appspot.com/10541079 TBR=zea@chromium.org Review URL: https://chromiumcodereview.appspot.com/10823061 git-svn-id: svn://svn.chromium.org/chrome/trunk/src@148812 0039d316-1c4b-4281-b951-d872f2087c98
generated by cgit v1.1 at 2025-02-12 14:55:42 +0000