GIT Repo Module

Submitted by SteveHeinsch on Tue, 09/22/2009 - 01:48

Hi Jamie/Joe/VM Team :)

I was wondering about the possibility of GIT support like you have for SVN. It would be great if my clients could set it up via vm as more and more are requesting it.

Some things I think would be nice: -Be able to point to some directory of theirs and Create a repo -Be able to generate ssh keys right there and have them merged into the /home/servername/.ssh/

Thanks for your thoughts, Steve

Status: 
Closed (fixed)

Comments

Submitted by JamieCameron on Tue, 09/22/2009 - 18:28

That's not a bad idea, as we use GIT ourselves for source code control.

However, isn't all GIT access via SSH? This wouldn't work so well for granting regular mailbox users access to the repository ..

Submitted by SteveHeinsch on Wed, 09/23/2009 - 15:28

Yes. The way I kind of envisioned it would be that a user sends the site admin their ssh key and there could be a place where the admin can append that key to /home/site/.ssh/authorized_keys. Just a textbox to paste the key into...click "merge"....something like that. I also believe GIT can be accessed using WebDAV but I haven't tried that. Would be best to stick to ssh I think as that is the most common use. I am nowhere near a GIT expert though :)

Submitted by SteveHeinsch on Thu, 09/24/2009 - 13:43

I dont think there needs to be a new user created does there? If the users key is added to /home/domain/.ssh/authorized_keys then they just have to use the domain name for username and their key? Since that username (site admin) already has SSH access? I could be wrong.

Submitted by JamieCameron on Thu, 09/24/2009 - 13:49

I guess it could be done that way .. but how then would Git know which user is making each commit? Or does it just trust the client for that?

Submitted by SteveHeinsch on Thu, 09/24/2009 - 14:03

By their local git client settings. You set the username and email and they show up in the commit messages.

git config --global user.name "myusername" git config --global user.email "myemail@mydomain.tld"

Those are global settings. If you want to change them per/repo you can

cd my_other_repo git config user.name "otherusername" git config user.email "otheremail@otherhost.tld"

sorry for the large text...I used pound signs at the beginning of each of those lines and it took them out and made the text big?

Submitted by JamieCameron on Thu, 09/24/2009 - 15:58

Ok, so it just trusts the client .. I will look into a GIT plugin, when I find the time :-)

Submitted by SteveHeinsch on Thu, 09/24/2009 - 16:34

What? You mean you're not just sitting around looking for things to do? LOL :)

Submitted by SteveHeinsch on Fri, 01/22/2010 - 11:58

Just a friendly bump :)

Submitted by JamieCameron on Fri, 01/22/2010 - 13:45

Sorry, haven't had time to work on GIT support yet..

Submitted by jcdenaes on Mon, 03/15/2010 - 11:04

another "friendly bump" ;)

Submitted by JamieCameron on Mon, 03/15/2010 - 12:36

Still haven't looked at this yet, sorry ..

Submitted by christefano on Thu, 05/20/2010 - 03:52 Pro Licensee

I'd be interested in this, too.

Submitted by pmajster on Sun, 05/23/2010 - 08:04

me too. this is only feature that i am still missing :(

Submitted by JamieCameron on Sun, 05/23/2010 - 13:20

Sorry, but I still haven't had a chance to look at this yet ..

Submitted by beat on Sun, 05/23/2010 - 14:27

Any goal date or goal revision number for GIT repo support ? :-)

“The trouble with not having a goal is that you can spend your life running up and down the field and never score” -Bill Copeland

Submitted by Vnuce on Wed, 06/09/2010 - 13:52

Any news regarding the Git repo module? It's a feature we'd really like to see in Virtualmin. We've been considering switching to Git for quite some time, but have been always putting it off due to the non existence of this module.

By the way, great product ;)

Submitted by sgrayban on Thu, 06/10/2010 - 10:34

I thought GIT was a clone of SVN - in any case is there even enough of a user base to add this ?

Submitted by runnerz on Sun, 07/25/2010 - 17:33

I'd be interested in this, too.

Submitted by shinji on Tue, 08/10/2010 - 18:08

This sounds like a great idea. I should add that git doesn't have to be over http. You can get away with just ssh. I had a git repo setup on another host and I opted to not make a http interface to it at all although it could be done.

Submitted by JamieCameron on Wed, 08/11/2010 - 14:49

If I was to do this module, it would almost certainly use http only in order to better fit in with Virtualmin .. that way each domain could have its own separate set of repos.

Submitted by shinji on Fri, 08/13/2010 - 21:20

Couldn't each domain have it's own set of repos anyways without http? You should be able to manage within Virtualmin along with the potential option for more public http access if it is enabled within the module. You could maybe define the domain where they are accessed if http is turned on or even define which ones are accessable via http.

A structure like this should work although adjust as needed since I don't know svn's current setup.

Primary domain: /home/user/git/repo /home/user/git/repo2

Sub domains /home/user/domains/sub-server/git/repo

Submitted by JamieCameron on Fri, 08/27/2010 - 18:56

You will all be happy to know that I have started work on a Git plugin, modeled after the SVN plugin. The plan is to allow HTTP access to repositories via DAV..

Submitted by beat on Fri, 08/27/2010 - 19:22

FANTASTIC NEWS, Jamie !

Yes, very happy ! Keep up the great work!

Hope permissions settings will be more flexible than for the svn settings (and svn settings improved too), so that user access can be controlled not globally, but on folders/subfolders basis within a repository. ;-)

Submitted by JamieCameron on Fri, 08/27/2010 - 19:36

I'm still working on it, but at the very least it will be possible to control who can access each repo..

Submitted by JamieCameron on Wed, 09/01/2010 - 02:10

I've created a first version of the Git plugin, which you can install as follows :

  1. Go to Webmin -> Webmin Configuration -> Webmin Modules, and install a new module from the URL http://download.webmin.com/download/plugins/virtualmin-git.wbm.gz
  2. Go to System Settings -> Features and Plugins, and check the box next to the "Git Repositories" plugin
  3. Select a domain that will use Git, click "Edit Virtual Server" and enable the "Allow Git repositories" feature
  4. Go to Services -> Git Repositories and create a new repo. Make sure you grant the admin user for the domain access to it
  5. Click the "Repository Commands" button to see instructions for checking files in and out

Please let me know if you find any bugs. This plugin will be added to the Virtualmin YUM / APT repositories shortly..

Submitted by pmajster on Thu, 09/09/2010 - 20:11

that's great news here! It spend some time, but I'm very glad. thanks a lot Jamie!!!

Submitted by izoox on Wed, 09/15/2010 - 02:29

Was this ever added to the repos? I can't seem to install it using yum. Since all my other packages are installed that way, I'd prefer to manage it that way.

Submitted by sgrayban on Thu, 09/16/2010 - 11:33

I don't see it in the debian repo either.

Submitted by sgrayban on Thu, 09/16/2010 - 11:43

I see a bug with git and yum on centos5 ...

yum install git gitweb
Setting up Install Process
No package git available.
No package gitweb available.
Nothing to do

There are no git core programs in centos5

No virtualmin git plugin either....

yum install wbm-virtualmin-git
Setting up Install Process
No package wbm-virtualmin-git available.
Nothing to do apt-get install webmin-virtualmin-git
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package webmin-virtualmin-git

Submitted by JamieCameron on Thu, 09/16/2010 - 11:49

Oh, it looks like git on CentOS 5 is in the EPEL repository only. You can enable it with :

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm

I will update the docs.

Submitted by sgrayban on Thu, 09/16/2010 - 15:59

Have you fixed the repo's yet ? I'm still not finding the new plugin.

Joe's picture
Submitted by Joe on Thu, 09/16/2010 - 21:41 Pro Licensee

I haven't rolled the module out yet, as I haven't had good bandwidth available for the task (I'm in the Yukon Territory right now). Should be able to do so in a day or two.

Submitted by sgrayban on Fri, 09/17/2010 - 00:55

What the hell you doing up there ? There is like nothing up there. I'm from Kelowna, BC and I know !!!

Submitted by rrhode on Fri, 09/17/2010 - 21:41

LoL

I know that isn't relevant, but I laughed so I had to post it. I was searching for this git thing since it was mentioned in the forum post and found my way here eventually. Sounds like exciting stuff. I have never used git but I think it sounds like maybe its a little less delicate than SVN and I might want to try it out. So thanks for all your hard work!

Submitted by rrhode on Sun, 09/19/2010 - 16:21

Well I tried it out. I was reading about Git on here http://wiki.eclipse.org/EGit/User_Guide and it sounds awesome.

I have it installed but had to use the URL to install the module (guessing because Joe is still wandering around in the boonies but thats cool) however I am getting some errors. I think this is probably the right place to post it since it seems related to this modules initial install config somehow.

I think it is because I have suexec enabled. I can't click the Browse Repositories button to use gitweb.

In the virtual server's Apache logs it says:

suexec policy violation: see suexec log for more details Premature end of script headers: gitweb.cgi

In the suexec log its saying this:

command not in docroot (/var/www/git/gitweb.cgi)

I found a site that says "suexec requires the CGI script to be under the server's DocumentRoot (not the VirtualHost DocumentRoot). It is permitted, however, for the VirtualHost DocumentRoot to be a symlink to a directory that appears under the real DocumentRoot."

I tried symlinking the /home/domain.com/public_html/git/gitweb.cgi to the one in /var/www/git/gitweb.cgi but no go.

Submitted by JamieCameron on Sun, 09/19/2010 - 18:26

This could happen if you don't have apache suexec compiled to run CGIs under /home.

The standard Virtualmin install using our scripts puts in a custom suexec that will work, but if you have done a manual Virtualmin install you may need to re-compile suexec.

Submitted by rrhode on Sun, 09/19/2010 - 22:24

It was an auto install I installed several months ago the last time I switched servers. This install has always been Virtualmin Pro as well.

As far as I know I haven't used any other custom suexec. However, perhaps I changed a setting some place. Is there any Virtualmin settings that could have caused this that you can think of? I am out of ideas. I couldn't get it to work.

Submitted by JamieCameron on Sun, 09/19/2010 - 23:08

Are you on a Debian / Ubuntu system, or CentOS / Redhat?

Submitted by christefano on Sun, 09/19/2010 - 23:59 Pro Licensee

After setting up a git repo, running git clone http://example.com/git/repo.git returns:

warning: remote HEAD refers to nonexistent ref, unable to checkout.

I think this means that the repo is empty and there's no master branch and therefore nothing can be cloned. Any ideas?

Submitted by sgrayban on Mon, 09/20/2010 - 01:24

You aren't using example.com as the domain to git ? Cause that is a EXAMPLE and not the domain to use.

Submitted by christefano on Mon, 09/20/2010 - 04:16 Pro Licensee

Thanks for your comment, sgrayban. No, I'm not actually using example.com. That's what I wrote in my comment for privacy reasons.

How are you remotely accessing git repos created by Virtualmin?

Joe's picture
Submitted by Joe on Mon, 09/20/2010 - 20:01 Pro Licensee

The suexec error is because the cgi isn't in /home. suexec will not follow symlinks outside of /home, for security reasons.

If you want to run that cgi out of /var/www you'll need to create a virtual server that does not use suexec.

Submitted by rrhode on Tue, 09/21/2010 - 20:03

Hmm, but the cgi is in /home/mydomain/public_html/git

I only tried to symlink it to the other one to try and fix the problem but it doesn't fix it.

I would prefer to use it wherever its intended to be used. Which appears to be /home/mydomain/public_html/git/gitweb.cgi. I have no idea why it is saying something about /var/www. I believe I am using all the settings that Virtualmin uses by default at this point for suexec and Git though I could be wrong since it doesn't work.

Submitted by JamieCameron on Tue, 09/21/2010 - 22:39

Yes, the CGI gets copied to /home/mydomain/public_html/git , so as long as suexec has been compiled to run scripts from /home it should work fine.

What does the command suexec -V output on your system?

Submitted by rrhode on Wed, 09/22/2010 - 02:37 

# suexec -V
 -D AP_DOC_ROOT="/home"
 -D AP_GID_MIN=100
 -D AP_HTTPD_USER="apache"
 -D AP_LOG_EXEC="/var/log/httpd/suexec.log"
 -D AP_SAFE_PATH="/usr/local/bin:/usr/bin:/bin"
 -D AP_UID_MIN=500
 -D AP_USERDIR_SUFFIX="public_html"

Submitted by JamieCameron on Wed, 09/22/2010 - 12:16

That looks fine to me ..

I just thought of another possible issue - the gitweb RPM for CentOS includes the file /etc/httpd/conf.d/git.conf that overrides the location of gitweb.cgi . Try deleting that file, and then restarting Apache.

Submitted by rrhode on Wed, 09/22/2010 - 13:34

That did the trick =) Thanks guys! Awesome as always.

Submitted by SteveHeinsch on Sat, 10/02/2010 - 04:20

I finally got around to testing this out and I must say I am quite impressed! Its very easy to set up and use and I know my customers will love this. I was skeptical about using the webdav instead of ssh with keys, but this works great.

I found a few minor bugs that I will submit separately.

Thank you again, Jaime.

Oh, do you know how to enable the post-receive hook? On the repo I have set up pre vm-git I have the post-receive hook set to serverowner:serverowner and 755 permissions, and inside it points to a simple php script that sends an email with the commit message. I would love to do this here, but can't figure out why its not working and I can't find any logs that mention an error. If you have any insight it would be awesome to get this working as we rely on these auto emails so others can be alerted and check the changes out.

Here is there post-receive hook I am using (~/git/testrepo.git/hooks/post-receive):

read oldRef newRef
git log $oldRef..$newRef --stat | php ~/git/testrepo.git/hooks/auto-email.php

and the auto-email.php script:

<?php

// Get Log
$stdin = @fopen('php://stdin', 'r');
$git_message = fread($stdin, 8192);
fclose($stdin);

$to = 'git@xxx.com';
$subject = 'Git: A New Commit was pushed to the xxx repository.';
$from = 'From: Git@xxx.com <no-reply@xxx.com>';

$message = "A new commit was pushed to the xxx Repository.\n\n";
$message .= "Details:\n";
$message .= $git_message . "\n\n";
$message .= "------------------------------------------------------\n";
$message .= "Please note that this does not mean that these changes\n";
$message .= "have been merged or pulled into any branch yet.";

mail($to, $subject, $message, $from);
?>

Submitted by JamieCameron on Sat, 10/02/2010 - 11:35

Sorry, but I haven't looked into post-receive hooks yet.

I'm not sure they would even be possible in this case, as Virtualmin sets up the git repo to be accessed via SVN, so there is nothing actually run on the repository host when a commit happens.

Submitted by SteveHeinsch on Thu, 10/07/2010 - 03:31

That is a real bummer. Guess we will have to stick to the old system for now. There are 11 developers on this project and everybody relies on those commit messages. I hope there is a way to get them to work!

Thanks.

Submitted by Issues on Thu, 10/21/2010 - 05:20

Automatically closed -- issue fixed for 2 weeks with no activity.