Junio C Hamano [Thu, 22 Jun 2006 09:18:51 +0000 (02:18 -0700)]
Merge branch 'ff/c99'
* ff/c99:
Remove all void-pointer arithmetic.
Change types used in bitfields to be `int's.
Don't use empty structure initializers.
Cast pointers to `void *' when used in a format.
Don't instantiate structures with FAMs.
Initialize FAMs using `FLEX_ARRAY'.
Remove ranges from switch statements.
Junio C Hamano [Thu, 22 Jun 2006 09:15:45 +0000 (02:15 -0700)]
Merge branch 'pb/config'
* pb/config:
git_config: access() returns 0 on success, not > 0
repo-config: Fix late-night bug
Read configuration also from $HOME/.gitconfig
Fix setting config variables with an alternative GIT_CONFIG
Support for extracting configuration from different files
Check and document the options to prevent mistakes.
When multiple recipients are given to git-send-email on the same
--cc line the code does not properly handle it.
Full and proper parsing of the email addresses so I can detect
which commas mean a new email address is more than I care to implement.
In particular this email address: "bibo,mao" <bibo.mao@intel.com>
must not be treated as two email addresses.
So this patch simply treats all commas in recipient lists as
an error and fails if one is given.
At the same time it documents that git-send-email wants multiple
instances of --cc specified on the command line if you want to
cc multiple recipients.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
Yakov Lerner [Thu, 22 Jun 2006 01:47:00 +0000 (04:47 +0300)]
Pass -DDEFAULT_GIT_TEMPLATE_DIR only where actually used.
Before this patch, -DDEFAULT_GIT_TEMPLATE_DIR was passed on compilation
command line to all and every .c file compiled. In fact the macro
is used by only one .c file, and unused by all other .c files.
Remove -DDEFAULT_GIT_TEMPLATE_DIR where unused. Follow the example of
exec_cmd.o. Pass -DDEFAULT_GIT_TEMPLATE_DIR only where actually used.
Signed-off-by: Yakov Lerner <iler.ml@gmail.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
Andre Noll [Wed, 21 Jun 2006 02:46:05 +0000 (04:46 +0200)]
object-refs: avoid division by zero
Currently, we don't check refs_hash_size size and happily call
lookup_object_refs() even if refs_hash_size is zero which leads to
a division by zero in hash_obj().
Signed-off-by: Andre Noll <maan@systemlinux.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Junio C Hamano [Tue, 20 Jun 2006 09:47:14 +0000 (02:47 -0700)]
checkout -f: do not leave untracked working tree files.
Earlier we did not consider untracked working tree files
"precious", but we have always considered them fair game to
clobber. These days, branch switching by read-tree is more
careful and tries to protect untracked working tree files. This
caused the following workflow to stop working:
Because the second checkout leaves F from the previous state as
untracked file in the working tree, the merge would fail, trying
to protect F from being clobbered.
This changes "git checkout -f" to remove working tree files that
are known to git in the switched-from state but do not exist in
the switched-to state, borrowing the same logic from "reset --hard".
David Woodhouse [Tue, 20 Jun 2006 14:38:13 +0000 (15:38 +0100)]
Log peer address when git-daemon called from inetd
When we run git-daemon from inetd, even with the --verbose option, it
doesn't log the peer address. That logic was only in the standalone
daemon code -- move it to the execute() function instead. Tested with
both IPv6 and Legacy IP clients, in both inetd and daemon mode.
Signed-off-by: David Woodhouse <dwmw2@infradead.org> Acked-by: Jon Loeliger <jdl@jdl.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
This bug was hidden by the "future-proofing" of the test. Sigh.
When neither GIT_CONFIG nor GIT_CONFIG_LOCAL is set, do not use NULL,
but $GIT_DIR/config. Instead of using $GIT_DIR/config when only
GIT_CONFIG_LOCAL is set. Sorry.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
Linus Torvalds [Tue, 20 Jun 2006 00:42:35 +0000 (17:42 -0700)]
Add "named object array" concept
We've had this notion of a "object_list" for a long time, which eventually
grew a "name" member because some users (notably git-rev-list) wanted to
name each object as it is generated.
That object_list is great for some things, but it isn't all that wonderful
for others, and the "name" member is generally not used by everybody.
This patch splits the users of the object_list array up into two: the
traditional list users, who want the list-like format, and who don't
actually use or want the name. And another class of users that really used
the list as an extensible array, and generally wanted to name the objects.
The patch is fairly straightforward, but it's also biggish. Most of it
really just cleans things up: switching the revision parsing and listing
over to the array makes things like the builtin-diff usage much simpler
(we now see exactly how many members the array has, and we don't get the
objects reversed from the order they were on the command line).
One of the main reasons for doing this at all is that the malloc overhead
of the simple object list was actually pretty high, and the array is just
a lot denser. So this patch brings down memory usage by git-rev-list by
just under 3% (on top of all the other memory use optimizations) on the
mozilla archive.
It does add more lines than it removes, and more importantly, it adds a
whole new infrastructure for maintaining lists of objects, but on the
other hand, the new dynamic array code is pretty obvious. The change to
builtin-diff-tree.c shows a fairly good example of why an array interface
is sometimes more natural, and just much simpler for everybody.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Junio C Hamano [Tue, 20 Jun 2006 01:25:21 +0000 (18:25 -0700)]
Restore SIGCHLD to SIG_DFL where we care about waitpid().
It was reported that under one implementation of socks client
"git clone" fails with "error: waitpid failed (No child processes)",
because "git" is spawned after setting SIGCHLD to SIG_IGN.
Arguably it may be a broken setting, but we should protect
ourselves so that we can get reliable results from waitpid() for
the children we care about.
This patch resets SIGCHLD to SIG_DFL in three places:
- connect.c::git_connect() - initiators of git native
protocol transfer are covered with this.
- daemon.c::main() - obviously.
- merge-index.c::main() - obviously.
There are other programs that do fork() but do not waitpid():
http-push, imap-send. upload-pack does not either, but in the
case of that program, each of the forked halves runs exec()
another program, so this change would not have much effect
there.
Junio C Hamano [Tue, 20 Jun 2006 00:01:35 +0000 (17:01 -0700)]
xdiff: minor changes to match libxdiff-0.21
This reformats the change 621c53cc082299eaf69e9f2dc0274547c7d87fb0
introduced to match what upstream author implemented in libxdiff-0.21
without changing any logic (hopefully ;-). This is to help keep
us in sync with the upstream.
Linus Torvalds [Mon, 19 Jun 2006 17:44:15 +0000 (10:44 -0700)]
Add specialized object allocator
This creates a simple specialized object allocator for basic
objects.
This avoids wasting space with malloc overhead (metadata and
extra alignment), since the specialized allocator knows the
alignment, and that objects, once allocated, are never freed.
It also allows us to track some basic statistics about object
allocations. For example, for the mozilla import, it shows
object usage as follows:
and the simpler allocator shaves off about 2.5% off the memory
footprint off a "git-rev-list --all --objects", and is a bit
faster too.
[ Side note: this concludes the series of "save memory in object storage".
The thing is, there simply isn't much more to be saved on the objects.
Doing "git-rev-list --all --objects" on the mozilla archive has a final
total RSS of 131498 pages for me: that's about 513MB. Of that, the
object overhead is now just 56MB, the rest is going somewhere else (put
another way: the fact that this patch shaves off 2.5% of the total
memory overhead, considering that objects are now not much more than 10%
of the total shows how big the wasted space really was: this makes
object allocations much more memory- and time-efficient).
I haven't looked at where the rest is, but I suspect the bulk of it is
just the pack-file loading. It may be that we should pack the tree
objects separately from the blob objects: for git-rev-list --objects, we
don't actually ever need to even look at the blobs, but since trees and
blobs are interspersed in the pack-file, we end up not being dense in
the tree accesses, so we end up looking at more pages than we strictly
need to.
So with a 535MB pack-file, it's entirely possible - even likely - that
most of the remaining RSS is just the mmap of the pack-file itself. We
don't need to map in _all_ of it, but we do end up mapping a fair
amount. ]
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Fix setting config variables with an alternative GIT_CONFIG
When setting a config variable, git_config_set() ignored the variables
GIT_CONFIG and GIT_CONFIG_LOCAL. Now, when GIT_CONFIG_LOCAL is set, it
will write to that file. If not, GIT_CONFIG is checked, and only as a
fallback, the change is written to $GIT_DIR/config.
Add a test for it, and also future-proof the test for the upcoming
$HOME/.gitconfig support.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
Florian Forster [Sun, 18 Jun 2006 15:18:08 +0000 (17:18 +0200)]
Change types used in bitfields to be `int's.
According to ANSI C99 bitfields are only defined for `signed int' and `unsigned
int'. This patch corrects the bitfield in the `msg_data_t' type from
`imap-send.c'.
Signed-off-by: Florian Forster <octo@verplant.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Florian Forster [Sun, 18 Jun 2006 15:18:07 +0000 (17:18 +0200)]
Don't use empty structure initializers.
Empty initializers for structures are not allowed in ANSI C99. This patch
removes such an initializer from `builtin-read-tree.c'. Since the struct was
static (and is therefore implicitely initialized to zero anyway) it wasn't
actually needed.
Signed-off-by: Florian Forster <octo@verplant.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Florian Forster [Sun, 18 Jun 2006 15:18:05 +0000 (17:18 +0200)]
Don't instantiate structures with FAMs.
Since structures with `flexible array members' are an incomplete datatype ANSI
C99 forbids creating instances of them. This patch removes such an instance
from `diff-lib.c' and replaces it with a pointer to a `struct
combine_diff_path'. Since all neccessary memory is allocated at once the number
of calls to `xmalloc' is not increased.
Signed-off-by: Florian Forster <octo@verplant.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Florian Forster [Sun, 18 Jun 2006 15:18:03 +0000 (17:18 +0200)]
Remove ranges from switch statements.
Though very nice and readable, the "case 'a'...'z':" construct is not ANSI C99
compliant. This patch unfolds the range in `quote.c' and substitutes the
switch-statement with an if-statement in `http-fetch.c' and `http-push.c'.
Signed-off-by: Florian Forster <octo@verplant.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Petr Baudis [Sat, 17 Jun 2006 23:23:58 +0000 (01:23 +0200)]
Support for extracting configuration from different files
Add $GIT_CONFIG environment variable whose content is used instead
of .git/config if set. Also add $GIT_CONFIG_LOCAL as a
forward-compatibility cue for whenever we will finally come to support]
global configuration files (properly).
Signed-off-by: Petr Baudis <pasky@suse.cz> Signed-off-by: Junio C Hamano <junkio@cox.net>
Paul Mackerras [Sun, 18 Jun 2006 23:25:16 +0000 (09:25 +1000)]
Fix PPC SHA1 routine for large input buffers
The PPC SHA1 routine had an overflow which meant that it gave
incorrect results for input buffers >= 512MB. This fixes it by
ensuring that the update of the total length in bits is done using
64-bit arithmetic.
Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Dennis Stosberg [Sun, 18 Jun 2006 20:33:21 +0000 (22:33 +0200)]
Make t8001-annotate and t8002-blame more portable
These two tests assume that "sed" will not modify the final line of a
stream if it does not end with a newline character. The assumption is
not true at least for FreeBSD and Solaris 9. FreeBSD's "sed" appends
a newline character; "sed" in Solaris 9 even removes the incomplete
final line. This patch makes the test use perl instead.
Signed-off-by: Dennis Stosberg <dennis@stosberg.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
Linus Torvalds [Sun, 18 Jun 2006 18:45:02 +0000 (11:45 -0700)]
Remove "refs" field from "struct object"
This shrinks "struct object" to the absolutely minimal size possible.
It now contains /only/ the object flags and the SHA1 hash name of the
object.
The "refs" field, which is really needed only for fsck, is maintained in
a separate hashed lookup-table, allowing all normal users to totally
ignore it.
This helps memory usage, although not as much as I hoped: it looks like
the allocation overhead of malloc (and the alignment constraints in
particular) means that while the structure size shrinks, the actual
allocation overhead mostly does not.
[ That said: memory usage is actually down, but not as much as it should
be: I suspect just one of the object types actually ended up shrinking
its effective allocation size.
To get to the next level, we probably need specialized allocators that
don't pad the allocation more than necessary. ]
The separation makes for some code cleanup, though, and makes the ref
tracking that fsck wants a clearly separate thing.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Rene Scharfe [Sun, 18 Jun 2006 13:25:33 +0000 (15:25 +0200)]
Make release tarballs friendlier to older tar versions
git-tar-tree adds an extended pax header to archives if its first
parameter points to a commit. It confuses older tars and isn't
very useful in the case of git anyway, so stop doing it.
Idea: Junio, implementation: Junio. I just wrote it up. :-)
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Rene Scharfe [Sun, 18 Jun 2006 10:58:01 +0000 (12:58 +0200)]
git-tar-tree: no more void pointer arithmetic
Noticed by Florian Forster: Use a char pointer when adding offsets,
because void pointer arithmetic is a GNU extension. Const'ify the
function arguments while we're at it.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Rene Scharfe [Sun, 18 Jun 2006 10:57:59 +0000 (12:57 +0200)]
git-tar-tree: documentation update
* add example on how to avoid adding a global extended pax header
* don't mention linux anymore, use git itself as an example instead
* update to v1.4.0 ;-)
* append missing :: to the examples
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Rene Scharfe [Sun, 18 Jun 2006 10:57:56 +0000 (12:57 +0200)]
git-tar-tree: Simplify write_trailer()
We can write the trailer in one or at most two steps; it will always
fit within two blocks. With the last caller of get_record() gone we
can get rid of it.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Detect changed prefix and/or changed build flags in the middle
of the build (or between 'make' and 'make install'), and if change
is detected, make sure all objects are compiled with same build
flags and same prefix, thus avoiding inconsistent/broken build.
[jc: removed otherwise unnecessary Makefile target to test the
change this patch introduces. ]
Signed-off-by: Yakov Lerner <iler.ml@gmail.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
When git-format-patch was converted to a builtin an appropriate call
to setup_ident was missed and thus git-format-patch -s fails because
it doesn't look up anything in the password file.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
Linus Torvalds [Sun, 18 Jun 2006 01:47:58 +0000 (18:47 -0700)]
Some more memory leak avoidance
This is really the dregs of my effort to not waste memory in git-rev-list,
and makes barely one percent of a difference in the memory footprint, but
hey, it's also a pretty small patch.
It discards the parent lists and the commit buffer after the commit has
been shown by git-rev-list (and "git log" - which already did the commit
buffer part), and frees the commit list entry that was used by the
revision walker.
The big win would be to get rid of the "refs" pointer in the object
structure (another 5%), because it's only used by fsck. That would require
some pretty major surgery to fsck, though, so I'm timid and did the less
interesting but much easier part instead.
This (percentually) makes a bigger difference to "git log" and friends,
since those are walking _just_ commits, and thus the list entries tend to
be a bigger percentage of the memory use. But the "list all objects" case
does improve too.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Linus Torvalds [Sun, 18 Jun 2006 01:26:18 +0000 (18:26 -0700)]
Move "void *util" from "struct object" into "struct commit"
Every single user actually wanted this only for commit objects, and we
have no reason to waste space on it for other object types. So just move
the structure member from the low-level "struct object" into the "struct
commit".
This leaves the commit object the same size, and removes one unnecessary
pointer from all other object allocations.
This shrinks memory usage (still at a fairly hefty half-gig, admittedly)
of "git-rev-list --all --objects" on the mozilla repo by another 5% in my
tests.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Linus Torvalds [Wed, 14 Jun 2006 23:45:13 +0000 (16:45 -0700)]
Shrink "struct object" a bit
This shrinks "struct object" by a small amount, by getting rid of the
"struct type *" pointer and replacing it with a 3-bit bitfield instead.
In addition, we merge the bitfields and the "flags" field, which
incidentally should also remove a useless 4-byte padding from the object
when in 64-bit mode.
Now, our "struct object" is still too damn large, but it's now less
obviously bloated, and of the remaining fields, only the "util" (which is
not used by most things) is clearly something that should be eventually
discarded.
This shrinks the "git-rev-list --all" memory use by about 2.5% on the
kernel archive (and, perhaps more importantly, on the larger mozilla
archive). That may not sound like much, but I suspect it's more on a
64-bit platform.
There are other remaining inefficiencies (the parent lists, for example,
probably have horrible malloc overhead), but this was pretty obvious.
Most of the patch is just changing the comparison of the "type" pointer
from one of the constant string pointers to the appropriate new TYPE_xxx
small integer constant.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Dennis Stosberg [Sat, 17 Jun 2006 15:02:45 +0000 (17:02 +0200)]
Make t4101-apply-nonl bring along its patches
Some versions of "diff" (e.g. on FreeBSD and older Linux systems) do
not support the "\ No newline at end of file" remark and are not
able to generate the patches needed for this test. This lets the
test fail, although git-apply is working perfectly. This patch adds
the pre-generated patches to t/t4100/ and makes the test use them.
Signed-off-by: Dennis Stosberg <dennis@stosberg.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
Linus Torvalds [Sun, 11 Jun 2006 17:57:35 +0000 (10:57 -0700)]
gitweb.cgi history not shown
This does:
- add a "rev.simplify_history" flag which defaults to on
- it turns it off for "git whatchanged" (which thus now has real
semantics outside of "git log")
- it adds a command line flag ("--full-history") to turn it off for
others (ie you can make "git log" and "gitk" etc get the semantics if
you want to.
Now, just as an example of _why_ you really really really want to simplify
history by default, apply this patch, install it, and try these two
command lines:
and it will show the old history of gitweb.cgi, even though it's not
relevant to the _current_ state of the name "gitweb.cgi"
NOTE NOTE NOTE! It will still actually simplify away merges that didn't
change anything at all into either child. That creates these bogus strange
discontinuities if you look at it with "gitk" (look at the --full-history
gitk output for git.c, and you'll see a few strange cases).
So the whole "--parent" thing ends up somewhat bogus with --full-history
because of this, but I'm not sure it's worth even worrying about. I don't
think you'd ever want to really use "--full-history" with the graphical
representation, I just give it as an example exactly to show _why_ doing
so would be insane.
I think this is trivial enough and useful enough to be worth merging into
the stable branch.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Florian Forster [Mon, 12 Jun 2006 08:31:57 +0000 (10:31 +0200)]
gitweb: Make the `blame' interface in gitweb optional.
Since `git-annotate' is an expensive operation to run it may be
desirable to deactivate this functionality. This patch introduces
the `gitweb.blame' option to git-repo-config and disables the blame
support by default.
Signed-off-by: Florian Forster <octo@verplant.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Florian Forster [Sun, 11 Jun 2006 15:45:19 +0000 (17:45 +0200)]
gitweb: Adding a `blame' interface.
This patch adds an interface for `git-blame' to `gitweb.cgi'.
Links to it are placed in `git_blob'.
Internally the code uses `git-annotate' because `git-blame's output
differs for files that have been renamed in the past. However, I like
the term `blame' better.
[jc: blame can be told to produce the compatible format btw...]
Signed-off-by: Florian Forster <octo@verplant.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Martin Langhoff [Mon, 12 Jun 2006 11:50:49 +0000 (23:50 +1200)]
cvsimport: keep one index per branch during import
With this patch we have a speedup and much lower IO when
importing trees with many branches. Instead of forcing
index re-population for each branch switch, we keep
many index files around, one per branch.
Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> Signed-off-by: Junio C Hamano <junkio@cox.net>
Martin Langhoff [Sun, 11 Jun 2006 08:12:20 +0000 (20:12 +1200)]
cvsimport: complete the cvsps run before starting the import
We now capture the output of cvsps to a tempfile, and then read it in.
cvsps 2.1 works quite a bit "in memory", and only prints its patchset
info once it has finished talking with cvs, but apparently retaining
all that memory allocation. With this patch, cvsps is finished and
reaped before cvsimport start working (and growing). So the footprint
of the whole process is much lower.
Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> Signed-off-by: Junio C Hamano <junkio@cox.net>
Martin Langhoff [Sun, 11 Jun 2006 08:12:09 +0000 (20:12 +1200)]
cvsimport: ignore CVSPS_NO_BRANCH and impossible branches
cvsps output often contains references to CVSPS_NO_BRANCH, commits
that it could not trace to a branch. Ignore that branch.
Additionally, cvsps will sometimes draw circular relationships
between branches -- where two branches are recorded as opening
from the other. In those cases, and where the ancestor branch
hasn't been seen, ignore it.
Signed-off-by: Martin Langhoff <martin@catalyst.net.nz> Signed-off-by: Junio C Hamano <junkio@cox.net>
Junio C Hamano [Wed, 14 Jun 2006 13:01:05 +0000 (06:01 -0700)]
fix git alias
When extra command line arguments are given to a command that
was alias-expanded, the code generated a wrong argument list,
leaving the original alias in the result, and forgetting to
terminate the new argv list.
Sean [Thu, 15 Jun 2006 21:26:21 +0000 (17:26 -0400)]
Add a "--notags" option for git-p4import.
P4import currently creates a git tag for every commit it imports.
When importing from a large repository too many tags can be created
for git to manage, so this provides an option to shut that feature
off if necessary.
Signed-off-by: Sean Estabrooks <seanlkml@sympatico.ca>
Junio C Hamano [Sat, 17 Jun 2006 05:33:28 +0000 (22:33 -0700)]
Merge git://git.bogomips.org/git-svn
* git://git.bogomips.org/git-svn: (25 commits)
git-svn: rebuild convenience and bugfixes
git-svn: svn (command-line) 1.0.x compatibility
git-svn: tests no longer fail if LC_ALL is not a UTF-8 locale
git-svn: bugfix and optimize the 'log' command
git-svn: Eliminate temp file usage in libsvn_get_file()
git-svn: fix several small bugs, enable branch optimization
git-svn: avoid creating some small files
git-svn: make the $GIT_DIR/svn/*/revs directory obsolete
git-svn: add support for Perl SVN::* libraries
git-svn: add 'log' command, a facsimile of basic `svn log'
git-svn: add UTF-8 message test
git-svn: add some functionality to better support branches in svn
git-svn: add --shared and --template= options to pass to init-db
git-svn: add --repack and --repack-flags= options
git-svn: minor cleanups, extra error-checking
git-svn: Move all git-svn-related paths into $GIT_DIR/svn
git-svn: support manually placed initial trees from fetch
git-svn: optimize --branch and --branch-all-ref
git-svn: --branch-all-refs / -B support
git-svn: support -C<num> passing to git-diff-tree
...
Eric Wong [Fri, 16 Jun 2006 01:48:22 +0000 (18:48 -0700)]
git-svn: bugfix and optimize the 'log' command
Revisions with long commit messages were being skipped, since
the 'git-svn-id' metadata line was at the end and git-log uses a
32k buffer to print the commits.
Also the last 'git-svn-id' metadata line in a commit is always
the valid one, so make sure we use that, as well.
Made the verbose flag work by passing the correct option switch
('--summary') to git-log.
Finally, optimize -r/--revision argument handling by passing
the appropriate limits to revision
Eric Wong [Thu, 15 Jun 2006 19:50:12 +0000 (12:50 -0700)]
git-svn: fix several small bugs, enable branch optimization
Share the repack counter between branches when doing
multi-fetch.
Pass the -d flag to git repack by default. That's the
main reason we will want automatic pack generation, to
save space and improve disk cache performance. I won't
add -a by default since it can generate extremely large
packs that make RAM-starved systems unhappy.
We no longer generate the .git/svn/$GIT_SVN_ID/info/uuid
file, either. It was never read in the first place.
Check for and create .rev_db if we need to during fetch (in case
somebody manually blew away their .rev_db and wanted to start
over. Mainly makes debugging easier).
Croak with $? instead of $! if there's an error closing pipes
Eric Wong [Tue, 13 Jun 2006 11:02:23 +0000 (04:02 -0700)]
git-svn: make the $GIT_DIR/svn/*/revs directory obsolete
This is a very intrusive change, so I've beefed up the tests
significantly. Added 'full-test' a target to the Makefile,
to test different possible configurations. This is intended
for maintainers only. Users should only be concerned with
'test' succeeding.
We now have a very simple custom database format for handling
mapping of svn revisions => git commits. Of course, we're
not really using it yet, either.
Also disabled automatic branch-finding on new trees for now.
It's too easily broken. revisions_eq() function should be
helpful for branch detection.
Also removed an extra assertion in fetch_cmd() that wasn't
correctly done. This bug was found by full-test.
Eric Wong [Mon, 12 Jun 2006 22:23:48 +0000 (15:23 -0700)]
git-svn: add support for Perl SVN::* libraries
This means we no longer have to deal with having bloated SVN
working copies around and we get a nice performance increase as
well because we don't have to exec the SVN binary and start a
new server connection each time.
Of course we have to manually manage memory with SVN::Pool
whenever we can, and hack around cases where SVN just eats
memory despite pools (I blame Perl, too). I would like to
keep memory usage as stable as possible during long fetch/commit
processes since I still use computers with only 256-512M RAM.
commit should always be faster with the SVN library code. The
SVN::Delta interface is leaky (or I'm not using it with pools
correctly), so I'm forking on every commit, but that doesn't
seem to hurt performance too much (at least on normal Unix/Linux
systems where fork() is pretty cheap).
fetch should be faster in most common cases, but probably not all.
fetches will be faster where client/server delta generation is
the bottleneck and not bandwidth. Of course, full-files are
generated server-side via deltas, too. Full files are always
transferred when they're updated, just like git-svnimport and
unlike command-line svn. I'm also hacking around memory leaks
(see comments) here by using some more forks.
I've tested fetch with http://, https://, file://, and svn://
repositories, so we should be reasonably covered in terms of
error handling for fetching.
Of course, we'll keep plain command-line svn compatibility as a
fallback for people running SVN 1.1 (I'm looking into library
support for 1.1.x SVN, too). If you want to force command-line
SVN usage, set GIT_SVN_NO_LIB=1 in your environment.
We also require two simultaneous connections (just like
git-svnimport), but this shouldn't be a problem for most
servers.
Less important commands:
show-ignore is slower because it requires repository
access, but -r/--revision <num> can be specified.
graft-branches may use more memory, but it's a
short-term process and is funky-filename-safe.
Eric Wong [Thu, 1 Jun 2006 09:35:44 +0000 (02:35 -0700)]
git-svn: add 'log' command, a facsimile of basic `svn log'
This quick feature should make it easy to look up svn log
messages when svn users refer to -r/--revision numbers.
The following features from `svn log' are supported:
--revision=<n>[:<n>] - is supported, non-numeric args are not:
HEAD, NEXT, BASE, PREV, etc ...
-v/--verbose - just maps to --raw (in git log), so
it's completely incompatible with
the --verbose output in svn log
--limit=<n> - is NOT the same as --max-count,
doesn't count merged/excluded commits
--incremental - supported (trivial :P)
New features:
--show-commit - shows the git commit sha1, as well
--oneline - our version of --pretty=oneline
Any other arguments are passed directly to `git log'
Eric Wong [Mon, 12 Jun 2006 22:53:13 +0000 (15:53 -0700)]
git-svn: add some functionality to better support branches in svn
New commands:
graft-branches - The most interesting command of the bunch. It
detects branches in SVN via various techniques (currently
regexes and file copies). It can be later extended to handle
svk and other properties people may use to track merges in svk.
Basically, merge tracking is not standardized at all in the SVN
world, and git grafts are perfect for dealing with this
situation.
Existing branch support (via tree matches) is only handled at
fetch time.
The following tow were originally implemented as shell scripts
several months ago, but I just decided to streamline things a
bit and added them to the main script.
multi-init - supports git-svnimport-like command-line syntax for
importing repositories that are layed out as recommended by the
SVN folks. This is a bit more tolerant than the git-svnimport
command-line syntax and doesn't require the user to figure out
where the repository URL ends and where the repository path
begins.
multi-fetch - runs fetch on all known SVN branches we're
tracking. This will NOT discover new branches (unlike
git-svnimport), so multi-init will need to be re-run (it's
idempotent).
Consider these three to be auxilliary commands (like
show-ignore, and rebuild) so their behavior won't receive as
much testing or scrutiny as the core commands (fetch and
commit).
Eric Wong [Wed, 24 May 2006 08:22:07 +0000 (01:22 -0700)]
git-svn: Move all git-svn-related paths into $GIT_DIR/svn
Since GIT_SVN_ID usage is probably going to become more
widespread <evil grin>, we won't run the chance of somebody
having a GIT_SVN_ID name that conflicts with one of the default
directories that already exist in $GIT_DIR (branches/tags).
Eric Wong [Thu, 4 May 2006 05:54:00 +0000 (22:54 -0700)]
git-svn: support manually placed initial trees from fetch
Sometimes I don't feel like downloading an entire tree again when
I actually decide a branch is worth tracking, so some users can
get around it more easily with this.
Eric Wong [Fri, 28 Apr 2006 10:51:16 +0000 (03:51 -0700)]
git-svn: optimize --branch and --branch-all-ref
By breaking the pipe read once we've seen a commit twice.
This should make -B/--branch-all-ref faster and usable on a
frequent basis.
We use topological order now for calling git-rev-list, and any
commit we've seen before should imply that all parents have been
seen (at least I hope that's the case for --topo-order).
Eric Wong [Sat, 3 Jun 2006 09:56:33 +0000 (02:56 -0700)]
git-svn: don't allow commit if svn tree is not current
If new revisions are fetched, that implies we haven't merged,
acked, or nacked them yet, and attempting to write the tree
we're committing means we'd silently clobber the newly fetched
changes.
Eric Wong [Sat, 10 Jun 2006 06:27:01 +0000 (23:27 -0700)]
git-svn: eol_cp corner-case fixes
If we read the maximum size of our buffer into $buf, and the
last character is '\015', there's a chance that the character is
'\012', which means our regex won't work correctly. At the
worst case, this could introduce an extra newline into the code.
We'll now read an extra character if we see '\015' is the last
character in $buf.
We also forgot to recalculate the length of $buf after doing the
newline substitution, causing some files to appeare truncated.
We'll do that now and force byte semantics in length() for good
measure.
Junio C Hamano [Sat, 10 Jun 2006 06:09:49 +0000 (23:09 -0700)]
shared repository: optionally allow reading to "others".
This enhances core.sharedrepository to have additionally
specify that read and exec permissions to be given to others as
well. It is useful when serving a repository via gitweb and
git-daemon that runs as a user outside the project group.
The configuration item can take the following values:
[core]
sharedrepository ; the same as "group"
sharedrepository = true ; ditto
sharedrepository = 1 ; ditto
sharedrepository = group ; allow rwx to group
sharedrepository = all ; allow rwx to group, allow rx to other
sharedrepository = umask ; not shared - use umask
It also extends "git init-db" to take "--shared=all" and friends
from the command line.
Junio C Hamano [Sat, 10 Jun 2006 05:07:23 +0000 (22:07 -0700)]
shared repository - add a few missing calls to adjust_shared_perm().
There were a few calls to adjust_shared_perm() that were
missing:
- init-db creates refs, refs/heads, and refs/tags before
reading from templates that could specify sharedrepository in
the config file;
- updating config file created it under user's umask without
adjusting;
- updating refs created it under user's umask without
adjusting;
- switching branches created .git/HEAD under user's umask
without adjusting.
This moves adjust_shared_perm() from sha1_file.c to path.c,
since a few SIMPLE_PROGRAM need to call repository configuration
functions which in turn need to call adjust_shared_perm().
sha1_file.c needs to link with SHA1 computation library which
is usually not linked to SIMPLE_PROGRAM.