How weld pull and weld push work¶
How “weld pull” does its stuff¶
Obviously, the code is the final word on what happens, but this is intended as reasonable background.
Remember, “weld pull” updates its idea of the bases, and then updates the seams in the weld “to match”.
The main code for this is inwelded/pull.py
The very short form¶
We make sure we have a current copy of the base (in .weld/bases
), copy
across the changes for each seam in that base to our weld, and commit with
an X-Weld-State: Merged
message.
- The local copy of the base is updated, but is not otherwise changed.
- The relevant seams in the weld are updated to match the equivalent directories in the base.
The short form¶
- Check it’s safe to do a
weld pull
- Update the copy of the base in
.weld/bases
- Find the last time
weld pull
orweld push
was done for this base - this is the synchronisation point. - Determine which seams have been added/removed/changed in comparison to the newly update base. If nothing has changed, then there is nothing to do.
- If there are deleted seams, delete them in the weld. If there are added seams, add them in the weld, by copying their files across. If there are changed seams, then replay the appropriate changes in the weld.
- Commit with an
X-Weld-State: Merged
message.
In detail¶
So we’re pulling a base
(You can also pull multiple bases at once, by giving multiple base names on the command line, or use
weld pull _all
to pull all bases, but these both just work by doing this whole sequence for each base in turn. Note that this can be more confusing, for instance if the Nth base requires remedial action to take to “finish” it, at which point you have to fix the problem, doweld finish
, and then give the original weld command again to pull the remaining bases.)(Pulling an individual seam would in theory be possible, but rather fiddly, and of questionable use anyway, so we’ll go with just pulling bases).
Given the name of a base:
Weld checks that there are no local changes in the weld - specifically, it runs
git status
in the weld’s base directory (the directory containing the.weld
directory). If there are any files in the weld that could be added withgit add
or committed withgit commit
, then it will refuse to proceed, suggesting that the user commit or stash the changes first.It also checks whether:
- the user is part way through an unfinished
weld pull
orweld push
- the weld could be updated with
git pull
(it looks at the remote repository to determine this) - the current branch is a weld-specific branch (starting with
weld-
).
and refuses to proceeed if any of those is true (this is essentially what
weld status
does, so you can do it beforehand as well).- the user is part way through an unfinished
It finds the last time that
weld pull
orweld push
was done for this base, by looking for the most recent commit with anX-Weld-State: Merged <base-name>
orX-Weld-State: Pushed <base-name>
message. If there isn’t one (i.e., this is the firstweld pull
orweld push
for this weld), then it uses theX-Weld-State: Init
commit instead.For
weld pull
we want to know the last time our base was synchronised with the weld as a whole. Since bothweld pull
andweld push
do this, we can use either as the relevant place to work from.Weld makes sure its copy of the base is up-to-date:
If it doesn’t yet have a clone of the base, it does:
$ cd .weld/bases $ git clone <base-repository> <base> $ cd <base> $ git pull
If it does have a clone of the base, it does:
$ cd .weld/bases/<base> $ git pull
In either case, it notes the HEAD commit of the base.
It determines which (if any) seams have been deleted, changed or added in the weld (with respect to the now up-to-date base). If all of those lists are empty, there is nothing to do, and the
weld pull
for this base is finished.It branches the weld. The branch point is the synchronisation commit that was found earlier (the last
weld pull
orweld push
commit, or else the Init commit).(The branch name used is chosen to be unique to this repository, and is currently of the form “weld-merge-<commit-id>-<index>”, where <commit-id> is the first 10 characters of the synchronisation commits SHA1 id, and <index> is chosen to make the branch unique in case that is not enough.)
It then:
- deletes any deleted seams
- modifies any modified seams
- adds any added seams
within that branch.
Deleting a seam is easy - it just means deleting the appropriate directory.
Adding a seam just copies the directory structure for that seam across from the base into the correct place in the weld.
Modifying a seam uses
git diff
to determine the appropriate changes in the base, and then replays them in the weld.Note
TODO It occurs to me that the technique use in
weld push
might be more efficient than this last, if it turns out to be usable - I’d need to think on this further. (Tibs)It writes
.weld/complete.py
and.weld/abort.py
, which can later be used by theweld finish
andweld abort
commands if necessary (and which will be deleted if theweld pull
of this base doesn’t need user interaction).It merges the original branch (typically
master
) onto this temporary branch. This will commonly “just work”, but if anything goes wrong, theweld pull
stops with a return code of 1 and a message of the form:<merge error message> Merge failed Either fix your merges and then do 'weld finish', or do 'weld abort' to give up.
If the merge onto the branch succeeded, or if the user fixes problems and then does
weld finish
, then thecomplete.py
script is run, which:- changes back to the original branch
- calculates the difference between this branch and the temporary branch on which we did our merge
- applies that patch to this original branch
- makes sure that any changed files are added to the git index (it does this over the entire weld, but that should be OK because nothing else should be changing the weld whilst we’re busy)
- commits this whole operation using an appropriate
X-Weld-State: Merged <base-name>
message. - deletes the
complete.py
andabort.py
scripts
At the moment, this doesn’t delete the temporary/working branch (which will show as a loop if you look in gitk). Future versions of weld may do so as part of the “complete” phase, but during the current active development it’s thought to be useful to leave the branch visible.
If the merge didn’t succeed, and the user chooses to do
weld abort
, then theabort.py
script is run, which:- switches back to the original branch
- deletes the temporary/working branch
- deletes the
complete.py
andabort.py
scripts
Also note that the weld-
branches are always meant to be local to the
current repository - they’re not meant to be pushed anywhere else.
Not having those “remotes/origin/weld-” branches¶
If you do a weld pull
and then do a git push
of the weld, in general
the transient branches will not be propagated to the weld’s remote.
However, if you clone directly from a “checked out” weld (rather than from a bare repository), then by default all branches are cloned, which is (a) untidy, and (b) mak cause future working branches to have the same name as earlier (remote) working branches.
If you have git version 1.7.10 or later, then you can instead clone a “working” weld using:
$ git clone --single-branch <weld-directory>
to retrieve (in this case) just master
(or use -b <branch
to name a
specific branch).
Of course, unfortunately, if you later do a git pull
, then the branches
will be fetched for you at that stage, so it’s not a perfect solution. But
then maybe you shouldn’t clone a “checked out” weld.
How “weld push” works¶
Obviously, the code is the final word on what happens, but this is intended as reasonable background.
Again, we’re only going to look at doing “weld push” on a single base - the
command line will take more than one base name, or the magic _all
, but
we’ll ignore that here.
The main code for this is inwelded/push.py
The very short form¶
We make sure we have a current copy of the base (in .weld/bases
), copy
across the changes for each seam in that base from our weld to the base,
commit them all as one change with an X-Weld-State: Pushed
message, and
push to the base’s origin. We also add an empty X-Weld-State: Pushed
commit in the weld, as a marker of when the weld push
happened.
- The base is updated to match its seams in the weld, and pushed to its remote.
- The weld is marked with when the push happened.
The short form¶
- Check it’s safe to do a
weld push
- Update the copy of the base in
.weld/bases
- Determine the last
weld push
for this base - For each seam, work out which files have changed (added, removed, changed)
in the weld, since that last
weld push
- Use
rsync
to make the files in (the corresponding directory in) the base match - Commit that in the base, and push it to the base’s remote
- Add a corresponding
X-Weld-State: Pushed
commit in the weld
Remember that only seams that are currently “named” in the weld are pushed, since they’re the only seams that are of interest “now” - if the user wanted to push changes to a seam that is not currently in use, then they should have done it when it was in use.
In detail¶
So doing weld push
for a given base name works as follows:
Weld checks that there are no local changes in the weld - specifically, it runs
git status
in the weld’s base directory (the directory containing the.weld
directory). If there are any files in the weld that could be added withgit add
or committed withgit commit
, then it will refuse to proceed, suggesting that the user commit or stash the changes first.It also checks whether:
- the user is part way through an unfinished
weld pull
orweld push
- the weld could be updated with
git pull
(it looks at the remote repository to determine this) - the current branch is a weld-specific branch (starting with
weld-
).
and refuses to proceeed if any of those is true (this is essentially what
weld status
does, so you can do it beforehand as well).- the user is part way through an unfinished
Weld makes sure its copy of the base is up-to-date:
If it doesn’t yet have a clone of the base, it does:
$ cd .weld/bases $ git clone <base-repository> <base> $ cd <base> $ git pull
If it does have a clone of the base, it does:
$ cd .weld/bases/<base> $ git pull
It finds the last time that
weld push
was done for this base, by looking for the most recent commit with anX-Weld-State: Pushed <base-name
message. If there isn’t one (i.e., this is the firstweld push
for this weld), then it uses theX-Weld-State: Init
commit instead.Why do we use the last
weld push
, and not the lastweld push
orweld pull
?Consider the following “pseudo git log”:
6 + change B to a file in base <fred> 5 o X-Weld-State: Merged <fred>/... 4 - a change to some irrelevant file(s) 3 + change A to a file in base <fred> 2 o X-Weld-State: Pushed <fred>/... 1 x some common commit
So we last did
weld pull
at commit 5, and the weld thus contains all the changes from base <fred> up to that point.However, we last did a
weld push
at commit 2, which means that changes 3 and 6 have still to be applied to base <fred>. But change 3 is before our lastweld pull
, so we definitely want the lastpush
.Note
Remember:
weld pull
updates the base from its remote, and then brings any changes therein into our weld. It does not propagate any changes in the weld back to the base.Weld looks up the current seams being used for this base. This tells it which directories (in the weld and in the base) it is interested in.
It looks up all of the changes in the weld since the synchronisation point (using
git log --one-line
) and remembers them.If there aren’t any, it has finished.
It trims out any
X-Weld-State
commits from that list, and remembers it.Again, if there are no changes (left), then it has finished.
As an aid during development (so this may go away later on), it tags the synchronisation commit, using a tag name of the form
weld-last-<base-name>-sync-<commit-id>
, where <commit-id> is the first 10 characters of its SHA1 commit id.In the base, it branches at the synchronisation point (remember,
X-Weld-State
commit messages record the equivalent commit id in the base as well), using a branch name of the form “weld-pushing-<commit-id>-<index>”.We then update the branch in the base:
For each seam that the base currently has in our weld:
- We use
git ls-files
in the appropriate seam in the weld to find out which files git is managing. - We do the same in the corresponding directory in the base.
- For files which the seam (in the weld) has, but the base does not, we
do
git rm
. We commit that change. - For all the other files in the seam, we just copy them over into the
base (actually, we use
rsync
). It is, of course, quite likely that many of them won’t have changed, but that’s OK. Then wegit add
all of the files we’ve copied, in the base, and commit that change as well.
Note
Any seams that are not in the weld are, by definition, not of interest to us. Even if they were in the weld at the last synchronisation point, the fact they aren’t now means we are not interested in any (possible) intermediate changes - if we cared about such changes, we should have done
weld push
then.- We use
We prepare a final commit message, and write out the
.weld/complete.py
and.weld/abort.py
files.We run the
complete.py
file to finish off ourweld push
. This:sets the merging indicator (touches a file in the
.weld/pushing
directory)in the base, if the merging indicator is not set, merges the original branch (normally “master”) into our working branch - this should just proceed with no problems
still in the base, merges that back into the original branch
still in the base, commits the change using the saved commit message
The commit message has a summary of the corresponding commits from the weld, as output by
git log --one-line
.If the user specified
weld push -edit
, then they get the chance to edit the message before it is used.notes the new HEAD commit id in the base
adds an
X-Weld-State: Pushed <base-name>/<commit-id>
commit to the weld, using that commit id (this is, of course, notionally an empty commit).deletes the merging indicator and the saved commit message.
If something did go wrong, then weld finish
just does that last item again
(which is why we need the merging indicator). weld abort
deletes the
working branch, and then deletes the merging indicator and saved commit
message.