Gitosis on Cygwin : Tips and Tweaks

In my last two posts, I blog about Git installation and Git Troubleshooting.

This time, I will cover some tweaks that you could do on Windows.

  • Hide unwanted users from Windows’s login screen
  • Change Gitosis path

Hide unwanted users from Windows’s login screen

I’m pretty sure that you don’t want the users created for Gitosis in your Windows’s login screen.

To Hide them, we will have to run a little script that will give them a special status in Windows’s Registry.


Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\SpecialAccounts\UserList]
"git"=dword:00000000
"cyg_server"=dword:00000000



Create a file with the extension .reg and double click on it. If you didn’t use the default username in Gitosis’s installation, you will have to adapt the script.

The result will be this :

Changed Gitosis path

One thing that I had to do, was to move Gitosis repository path. I installed Gitosis in the default home of the user git. (/home/git/…) The problem was that I didn’t have enough free space on that hard drive.

To solve that problem, I moved the home folder of git to my “D” drive.

All the magic start by editing : /etc/passwd logged with you Administrator user. You will have to change the path (/home/git) to a new path. In my case, I changed it to : /cygdrive/d/gitosis-repo


$ cat /etc/passwd
SYSTEM:*:18:544:,S-1-5-18::
LocalService:*:19:544:U-NT AUTHORITY\LocalService,S-1-5-19::
NetworkService:*:20:544:U-NT AUTHORITY\NetworkService,S-1-5-20::
Administrators:*:544:544:,S-1-5-32-544::
Administrator:unused:500:513:U-bob-PC\Administrator,S-1-5-21-2630779338-38
2115681-2868927028-500:/home/Administrator:/bin/bash
Guest:unused:501:513:U-bob-PC\Guest,S-1-5-21-2630779338-382115681-28689270
28-501:/home/Guest:/bin/bash
HomeGroupUser$:unused:1005:513:HomeGroupUser$,U-bob-PC\HomeGroupUser$,S-1-
5-21-2630779338-382115681-2868927028-1005:/home/HomeGroupUser$:/bin/bash
bob:unused:1003:513:U-bob-PC\bob,S-1-5-21-2630779338-382115681-28689
27028-1003:/home/bob:/bin/bash
sshd:unused:1006:513:sshd privsep,U-bob-PC\sshd,S-1-5-21-2630779338-382115
681-2868927028-1006:/var/empty:/bin/false
cyg_server:unused:1007:513:Privileged server,U-bob-PC\cyg_server,S-1-5-21-
2630779338-382115681-2868927028-1007:/var/empty:/bin/false
git:unused:1008:513:U-bob-PC\git,S-1-5-21-2630779338-382115681-2868927028-
1008:/cygdrive/d/gitosis-repo/:/bin/bash

After that, you have to log using git user : ssh git@localhost . It will create new profile in your new home folder.

Now, you have to moved the folder from your last home folder (/home/git) to this folder.

The important part now, it’s to recreate the symbolic link to gitosis.conf.

Use the command : ln -s TARGET LINK . In my case it was : ln -s /cygdrive/d/gitosis-repo/repositories/gitosis-admin.git/gitosis.conf .gitosis.conf


$ ls -la
total 30
drwxr-xr-x+ 1 git    None 4096 2010-09-11 15:34 .
drwxrwxrwx+ 1 jerabi None 4096 2010-09-11 15:32 ..
-rw-------  1 git    None  673 2010-09-11 15:38 .bash_history
-rwxr-xr-x  1 git    None 1150 2010-09-06 20:24 .bash_profile
-rwxr-xr-x  1 git    None 3754 2010-09-06 20:24 .bashrc
-rw-r--r--  1 git    None   55 2010-09-06 21:36 .gitconfig
lrwxrwxrwx  1 git    None   68 2010-09-11 15:32 .gitosis.conf -> /cygdrive/d/gitosis-repo/repositories/gitosis-admin.git/gitosis.conf
-rwxr-xr-x  1 git    None 1461 2010-09-06 20:24 .inputrc
drwx------+ 1 git    None 4096 2010-09-15 21:12 .ssh
drwxr-xr-x+ 1 git    None    0 2010-09-15 21:13 gitosis
drwxr-xr-x+ 1 git    None    0 2010-09-15 21:13 repositories

It was really useful for me, and I hope that it can help you too.

You can follow me on Twitter

Hosting Git Repository Server Gitosis on Windows 7 : Troubleshooting

If you have problems installing Gitosis on Windows, I suggest that you read that guide : : Git Repository Server Gitosis on Win7 (100% working), maybe you will be able to find what you missed.

In this guide, I’ll describe raw problems that I found with the solution. Before I came with this guide, I try to install Gitosis on 3 computers and 4 Virtualbox installation by follow tutorial that I found. I kept the errors that I made and I’ll explain that I found in this guide.

To save you lot of trouble, be sure that you put this line “loglevel = DEBUG” in gitosis.conf .

The errors are not shown in order.

If you can’t connect by ssh, be sure that you open the sh port in your firewall. By default it’s the port 22.


$ gitosis-init < id_rsa.pub
2 [main] python 66492 C:\cygwin\bin\python.exe: *** fatal error - unable t o remap
 \\?\C:\cygwin\lib\python2.6\lib-dynload\time.dll to same address as pare nt: 0x360000 != 0x380000 
2 [main] python 30908 fork: child 66492 - died waiting for dll loading, er rno 11 
Traceback (most recent call last): File "/usr/bin/gitosis-init", line 8, in 
 load_entry_point('gitosis==0.2', 'console_scripts', 'gitosis-init')() File "/usr/lib/python2.6/site-packages/
gitosis-0.2-py2.6.egg/gitosis/app.py", line 24, in run return app.main() File "/usr/lib/python2.6/site-packages/
gitosis-0.2-py2.6.egg/gitosis/app.py", line 38, in main self.handle_args(parser, cfg, options, args) File 
"/usr/lib/python2.6/site-packages/gitosis-0.2-py2.6.egg/gitosis/init.py", line 136, in handle_args
 user=user, File "/usr/lib/python2.6/site-packages/gitosis-0.2-py2.6.egg/gitosis/init.py", line 75, in
 init_admin_repository template=resource_filename('gitosis.templates', 'admin') File "/usr/lib/python2.6/
site-packages/gitosis-0.2-py2.6.egg/gitosis/repositor y.py", line 51, in init close_fds=True, File 
"/usr/lib/python2.6/subprocess.py", line 480, in call return Popen(*popenargs, **kwargs).wait() 
File "/usr/lib/python2.6/subprocess.py", line 633, in __init__ errread, errwrite) File "/usr/lib/python2.6/
subprocess.py", line 1049, in _execute_child self.pid = os.fork() OSError: 
[Errno 11] Resource temporarily unavailable 

If you had this error, it's because you installed Python 2.6 instead of 2.5. Reinstall Python 2.5 and Setuptool for Python 2.5.


$ git clone git@localhost:notexist.git
Initialized empty Git repository in /home/bob/notexist/.git/
DEBUG:gitosis.serve.main:Got command "git-upload-pack 'notexist.git'"
DEBUG:gitosis.access.haveAccess:Access check for 'bob' as 'writable' on 'notexist.git'...
DEBUG:gitosis.access.haveAccess:Stripping .git suffix from 'notexist.git', new v
alue 'notexist'
DEBUG:gitosis.group.getMembership:found 'bob' in 'gitosis-admin'
DEBUG:gitosis.group.getMembership:found 'bob' in 'first_repo'
DEBUG:gitosis.access.haveAccess:Access check for 'bob' as 'writeable' on 'notexi
st.git'...
DEBUG:gitosis.access.haveAccess:Stripping .git suffix from 'notexist.git', new v
alue 'notexist'
DEBUG:gitosis.group.getMembership:found 'bob' in 'gitosis-admin'
DEBUG:gitosis.group.getMembership:found 'bob' in 'first_repo'
DEBUG:gitosis.access.haveAccess:Access check for 'bob' as 'readonly' on 'notexis
t.git'...
DEBUG:gitosis.access.haveAccess:Stripping .git suffix from 'notexist.git', new v
alue 'notexist'
DEBUG:gitosis.group.getMembership:found 'bob' in 'gitosis-admin'
DEBUG:gitosis.group.getMembership:found 'bob' in 'first_repo'
ERROR:gitosis.serve.main:Repository read access denied
fatal: The remote end hung up unexpectedly

bob@bob-PC ~
$

The error is that the repository doesn't exist.


$ git push origin master:refs/heads/master
ERROR:gitosis.serve.main:Repository read access denied
fatal: The remote end hung up unexpectedly

There could be multiple reason for that one. Mostly is because you don't have the permission to write. Start by looking into gitosis.conf if your user have the permission for the repository. Also check if you put your ssh-rsa key into keydir. There another possible reason, and it was really hard to find. Check the file /home/git/.ssh/authorized_keys . One error that I made was to add my Windows user into this file and after that I add my key into keydir. The result was when I pushed, Gitosis found my key, but it didn't have the command : " command="gitosis-serve xxx" ". If you have twice your key in that file, remove the line that wasn't generated by Gitosis.

Here it's what authorized_keys look like after adding key into keydir.


$ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf2jt0E6wpOj2iUoJttpvedR3jHbu/LizVkzDWFmG/
nuwZltnhwz1uCk7ArK0LeRlJFE25A/KS2OGci6xTh9MeZ5E1UHEYhR83HF98KLp64QmzwSwL6LWOlDm1
Q5fYm4jCdkXs/C0+3zvdxiza5gUBwaBSEpCd3gtWiIMv2T+Vi8O73YlIITMrYAA3gzh1HBie9xjSiZRI
TDDiQ3kFUB+aUKkwuprZtmm1PArTUU16OPUjQbP31rsJp6j8oGZVNW+pCiL91lTvVGCGmqjBqXuT4FMG
3/xIBVWN1seflEp8U1HL6ZScvkXLCStPPOecKi5Q562bDeK26VC+3kXnCUgZ git@bob-PC
### autogenerated by gitosis, DO NOT EDIT
command="gitosis-serve bob",no-port-forwarding,no-X11-forwarding,no-agent-forwar
ding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDG2PKZmEwo53Rlm1LvSD52slhu+sTa
UUprTi3nfGhhYY3g1QAaonl6EWt4kZpD0U043f11TH8bJEwhytWo6EwxMsj9qfXcbORx2bj7zvNlHxTx
Akaz0i5vVkHCuv18ykexqcgNULxenMz6LaK9MY0eqP1ZNeWZkJVY1fSSLydv58zTP/ap3SC1H7feSdeF
5Hs9oZCvTR9J7A2KCjKlp5R0dC2EUeYmPGY0H0hmQnU4aGa9bxVq8iMfgzYkopubrpmXWvtMMgE3la+q
MskeHfdF7b0E9APaSnzTtjK2YMt52/FnPBF/99UvVjt9B9yOV3fsUgIGF1Ks5mjUlZRHX6VJ bob@bob
-PC
command="gitosis-serve git@bob-PC",no-port-forwarding,no-X11-forwarding,no-agent
-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf2jt0E6wpOj2iUoJttpved
R3jHbu/LizVkzDWFmG/nuwZltnhwz1uCk7ArK0LeRlJFE25A/KS2OGci6xTh9MeZ5E1UHEYhR83HF98K
Lp64QmzwSwL6LWOlDm1Q5fYm4jCdkXs/C0+3zvdxiza5gUBwaBSEpCd3gtWiIMv2T+Vi8O73YlIITMrY
AA3gzh1HBie9xjSiZRITDDiQ3kFUB+aUKkwuprZtmm1PArTUU16OPUjQbP31rsJp6j8oGZVNW+pCiL91
lTvVGCGmqjBqXuT4FMG3/xIBVWN1seflEp8U1HL6ZScvkXLCStPPOecKi5Q562bDeK26VC+3kXnCUgZ
git@bob-PC

bob@bob-PC /home/git/.ssh
$


$ git clone git@localost:home/git/repositories/gitosis-admin.git
Initialized empty Git repository in /home/survivant/from_ge/gitosis-admin/.git/
The authenticity of host 'ge (192.168.0.13)' can't be established.
RSA key fingerprint is 0c:fb:0b:70:62:1a:7a:2d:15:a8:36:25:6b:11:2e:09.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localost,192.168.0.13' (RSA) to the list of known hosts.
git@localost's password:
Permission denied, please try again.
git@localost's password:
Permission denied, please try again.
git@localost's password:
Permission denied (publickey,password,keyboard-interactive).
fatal: The remote end hung up unexpectedly

You will obtain this error when you didn't put the user's key into keydir.


bob@bob-PC ~/git_clone/first_repo
$ git push origin master:refs/heads/master
fatal: 'first_repo.git' does not appear to be a git repository
fatal: The remote end hung up unexpectedly

I obtain that error message because I didn't add : [group first_repo] in gitosis.conf


win7user@win7user-PC ~/git_clone/first_repo
$ git push
fatal: No destination configured to push to.

You need to add "origin" in you git repository. Use that command :

win7user@win7user-PC ~/git_clone/first_repo
$ git remote add origin git@localhost:first_repo.git

Another error is when you push a new gitosis.conf version and not still not able to push in your repositories. Be sure that you change the permission on the files :


git@bob-PC ~
$ chmod 755 /home/git/repositories/gitosis-admin.git/hooks/ -R

git@bob-PC ~
$ chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update

git@bob-PC ~
$

I really hope that can help your troubleshooting.

You can follow me on Twitter

Posted in Uncategorized | 1 Reply

Hosting Git Repository Server Gitosis on Windows 7 (100% working)

If you are here, it’s because you’re lost or you have problems installing a git repository server on Windows. I really hope it’s for the second reason :) .

I choose the wrote this blog even if I know there are a lot of blogs about this subject, because none of them worked for me. Before I came with this guide, I try to install Gitosis on 3 computers and 4 Virtualbox installation by follow tutorial that I found.

I’ll keep this blog as simple as possible. You won’t need to understand the steps that we will do (It won’t hurt, if you try). I’ll give you the steps to do it right the first time and do a second blog about troubleshooting : Git Repository Server Gitosis on Win7 : Troubleshooting.

You will start by downloading and installing/configuring your Windows computer and after that we will configure Gitosis. I’ll do a quick overview how to setup your client computer (could be on the same computer) to connect to your git repository on Windows. To keep it simple, I’ll call that computer your “server” or “git server”.

Downloading software needed

You will need to download these softwares on your server :

  • Cygwin : Cygwin is a Linux-like environment for Windows : download from here
  • Setuptools for cygwin : Allow us to install Python applications (Gitosys) easily : download from here or here and be sure to select the 2.5 version (.egg)

Before installing these softwares, we have to open Windows Firewall. If you have another firewall, be sure to apply the same change that we will do.

I found a really good guide how to do that, so I’ll give credit to the guy who wrote it. Just follow this guide, step by step, but at the end just change the port for 22 instead of 80 (you can change the description HTTP for SSHD). Go here

Cygwin’s installation

Launch setup.exe . You will have to add these applications. (Enter there name in the search box and select them)

python -> python 2.5.x (Don’t select versions over 2.5.x)

Python

ssh -> openssh

ssh

git -> git , gitgui, git-completion

git

vim -> vim

vim

When the installation is completed, run Cygwin with the Administrator rights. (Starts menu, Cygwin, Cygwin Bash (right click)

Run as Administrator

You must see this screen

Cygwin bash

Configuring Git Server

GOOD. The fun starts here. We have to properly setup the server in Cygwin before installing Gitosis. We will configure Python and setup a sshd server (Gitosis use ssh). In the next steps, you will see that I used a user named : “bob”, in your case it will be your Windows account.

Installing Setuptools

Go to the folder where you download the application. In my case I choose the default folder c:\Users\bob\Downloads


bob@bob-PC ~
$ cd ../../cygdrive/c/Users/bob/Downloads/

bob@bob-PC /cygdrive/c/Users/bob/Downloads
$

bob@bob-PC /cygdrive/c/Users/bob/Downloads
$ ./setuptools-0.6c11-py2.5.egg

The output will look like that :


$ ./setuptools-0.6c11-py2.5.egg
Processing setuptools-0.6c11-py2.5.egg
Copying setuptools-0.6c11-py2.5.egg to /usr/lib/python2.5/site-packages
Adding setuptools 0.6c11 to easy-install.pth file
Installing easy_install script to /usr/bin
Installing easy_install-2.5 script to /usr/bin

Installed /usr/lib/python2.5/site-packages/setuptools-0.6c11-py2.5.egg
Processing dependencies for setuptools==0.6c11
Finished processing dependencies for setuptools==0.6c11

bob@bob-PC /cygdrive/c/Users/bob/Downloads
$


bob@bob-PC /cygdrive/c/Users/bob/Downloads
$ chmod +r /usr/lib/python2.5/ -R

bob@bob-PC /cygdrive/c/Users/bob/Downloads
$

Configuring SSHD

We need to create a ssh-rsa key for your user. If you enter a passphrase, be sure to remember it. I used the default options for my setup.


$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/bob/.ssh/id_rsa):
Created directory '/home/bob/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/bob/.ssh/id_rsa.
Your public key has been saved in /home/bob/.ssh/id_rsa.pub.
The key fingerprint is:
cc:85:b2:99:ee:ca:41:eb:be:b7:d9:ee:1b:a6:46:20 bob@bob-PC
The key's randomart image is:
The key's randomart image is:
+--[ RSA 2048]----+
|        o. .oo o=|
|       +  . .=*=o|
|        o   +.+=+|
|       .   = .. o|
|        S . o  . |
|           E     |
|                 |
|                 |
|                 |
+-----------------+

bob@bob-PC /cygdrive/c/Users/bob/Downloads
$ cd

bob@bob-PC ~
$ cd .ssh/

bob@bob-PC ~/.ssh
$ cat id_rsa.pub > authorized_keys

bob@bob-PC ~/.ssh
$

We need to copy our public key to a temporary folder, because we will need it later.


bob@bob-PC ~/.ssh
$ mkdir ../../../cygdrive/c/temp

bob@bob-PC ~/.ssh
$ cp id_rsa.pub ../../../cygdrive/c/temp/bob.pub

bob@bob-PC ~/.ssh
$

The last step in this section : Setup SSHD service.

You will have few questions to answer. I put the answer after each questions in the description below.


bob@bob-PC ~/.ssh
$ ssh-host-config
*** Info: Generating /etc/ssh_host_key
*** Info: Generating /etc/ssh_host_rsa_key
*** Info: Generating /etc/ssh_host_dsa_key
*** Info: Creating default /etc/ssh_config file
*** Info: Creating default /etc/sshd_config file
*** Info: Privilege separation is set to yes by default since OpenSSH 3.3.
*** Info: However, this requires a non-privileged account called 'sshd'.
*** Info: For more info on privilege separation read /usr/share/doc/openssh/READ
ME.privsep.
*** Query: Should privilege separation be used? (yes/no) yes
*** Info: Note that creating a new user requires that the current account have
*** Info: Administrator privileges.  Should this script attempt to create a
*** Query: new local account 'sshd'? (yes/no) yes
*** Info: Updating /etc/sshd_config file


*** Warning: The following functions require administrator privileges!

*** Query: Do you want to install sshd as a service?
*** Query: (Say "no" if it is already installed as a service) (yes/no) yes
*** Query: Enter the value of CYGWIN for the daemon: [] ntsec tty
*** Info: On Windows Server 2003, Windows Vista, and above, the
*** Info: SYSTEM account cannot setuid to other users -- a capability
*** Info: sshd requires.  You need to have or to create a privileged
*** Info: account.  This script will help you do so.

*** Info: You appear to be running Windows 2003 Server or later.  On 2003
*** Info: and later systems, it's not possible to use the LocalSystem
*** Info: account for services that can change the user id without an
*** Info: explicit password (such as passwordless logins [e.g. public key
*** Info: authentication] via sshd).

*** Info: If you want to enable that functionality, it's required to create
*** Info: a new account with special privileges (unless a similar account
*** Info: already exists). This account is then used to run these special
*** Info: servers.

*** Info: Note that creating a new user requires that the current account
*** Info: have Administrator privileges itself.

*** Info: No privileged account could be found.

*** Info: This script plans to use 'cyg_server'.
*** Info: 'cyg_server' will only be used by registered services.
*** Query: Do you want to use a different name? (yes/no) no
*** Query: Create new privileged user account 'cyg_server'? (yes/no) yes
*** Info: Please enter a password for new user cyg_server.  Please be sure
*** Info: that this password matches the password rules given on your system.
*** Info: Entering no password will exit the configuration.
*** Query: Please enter the password:
*** Query: Reenter:

*** Info: User 'cyg_server' has been created with password 'encore00'.
*** Info: If you change the password, please remember also to change the
*** Info: password for the installed services which use (or will soon use)
*** Info: the 'cyg_server' account.

*** Info: Also keep in mind that the user 'cyg_server' needs read permissions
*** Info: on all users' relevant files for the services running as 'cyg_server'.

*** Info: In particular, for the sshd server all users' .ssh/authorized_keys
*** Info: files must have appropriate permissions to allow public key
*** Info: authentication. (Re-)running ssh-user-config for each user will set
*** Info: these permissions corrently. [Similary restrictions apply, for
*** Info: instance, for .rhosts files if the rshd server is running, etc].


*** Info: The sshd service has been installed under the 'cyg_server'
*** Info: account.  To start the service now, call `net start sshd' or
*** Info: `cygrunsrv -S sshd'.  Otherwise, it will start automatically
*** Info: after the next reboot.

*** Info: Host configuration finished. Have fun!

bob@bob-PC ~/.ssh
$

Now we start the service.


bob@bob-PC ~/.ssh
$ net start sshd
The CYGWIN sshd service is starting.
The CYGWIN sshd service was started successfully.

bob@bob-PC ~/.ssh
$

Let’s test it. We will connect and exit. You must have something like describe below.


$ ssh localhost
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is 16:38:f8:f2:f7:81:53:7c:cf:c3:03:c0:45:0d:7e:51.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.

bob@bob-PC ~
$ exit
logout
Connection to localhost closed.

bob@bob-PC ~/.ssh
$

Oh ya, the first big part is completed. The next part is Gitosis installation.

Gitosis installation

Now we will download and install Gitosis.


bob@bob-PC ~/.ssh
$ cd

bob@bob-PC ~
$ mkdir sources && cd sources

bob@bob-PC ~/sources
$

bob@bob-PC ~/sources
$ git clone git://eagain.net/gitosis.git
Initialized empty Git repository in /home/bob/sources/gitosis/.git/
remote: Counting objects: 614, done.
remote: Compressing objects: 100% (183/183), done.
remote: Total 614 (delta 434), reused 594 (delta 422)
Receiving objects: 100% (614/614), 93.82 KiB, done.
Resolving deltas: 100% (434/434), done.

bob@bob-PC ~/sources
$ cd gitosis/

bob@bob-PC ~/sources/gitosis
$ python setup.py install
running install
running bdist_egg
running egg_info
creating gitosis.egg-info
writing requirements to gitosis.egg-info/requires.txt
....
Using /usr/lib/python2.5/site-packages/setuptools-0.6c11-py2.5.egg
Finished processing dependencies for gitosis==0.2

bob@bob-PC ~/sources/gitosis
$

Gitosis is a service that will allow us to create multiple git repositories. There are other alternatives to Gitosis like MSysGit, but Gitosis is a more complete solution. You only need one unix account to handle your multiple git users with Gitosis, instead of having one account by user.

Let’s create that git user. I’ll create a user named “git” with a password “userspassword”. It will create a Windows user.


bob@bob-PC ~/sources/gitosis
$ net user git userspassword /add
The command completed successfully.

bob@bob-PC ~/sources/gitosis
$

We need the next line to add that new Windows’s user to Cygwin.


bob@bob-PC ~/sources/gitosis
$ mkpasswd -l -u git >> /etc/passwd

bob@bob-PC ~/sources/gitosis
$

We have to connect through ssh using this new user : git to continue Gitosis’s configuration


bob@bob-PC ~/sources/gitosis
$ ssh git@localhost
git@localhost's password:
Could not chdir to home directory /home/git: No such file or directory
Copying skeleton files.
These files are for the user to personalise their cygwin experience.

They will never be overwritten nor automatically updated.

`./.bashrc' -> `/home/git//.bashrc'
`./.bash_profile' -> `/home/git//.bash_profile'
`./.inputrc' -> `/home/git//.inputrc'

git@bob-PC ~
$

We have to setup git config for our user.


git@bob-PC ~
$ git config --global user.name "git User"

git@bob-PC ~
$ git config --global user.email you@ca.ca

git@bob-PC ~
$

We need to create a ssh-rsa key.


git@bob-PC ~
$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/git/.ssh/id_rsa):
Created directory '/home/git/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/git/.ssh/id_rsa.
Your public key has been saved in /home/git/.ssh/id_rsa.pub.
The key fingerprint is:
06:c8:e8:ee:39:96:f7:b6:48:5f:df:4f:ac:82:d4:43 git@bob-PC
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|   o .           |
|  . o .          |
| .     .  E      |
|  .     So       |
| .     .. o  .   |
|  ...  ... .  o  |
| .+o.o...... o   |
| .oo.o+.  ..o..  |
+-----------------+

git@bob-PC ~
$

git@bob-PC ~
$ cd .ssh/

git@bob-PC ~/.ssh
$ cat id_rsa.pub > authorized_keys

git@bob-PC ~/.ssh
$

Setup Gitosis

At this point, we have installed Cygwin, sshd, setup a new user : git and clone Gitosis repository. We need to setup Gitosis using git user.


git@bob-PC ~/.ssh
$ cd

git@bob-PC ~
$ gitosis-init < .ssh/id_rsa.pub
Initialized empty Git repository in /home/git/repositories/gitosis-admin.git/
Reinitialized existing Git repository in /home/git/repositories/gitosis-admin.git/

git@bob-PC ~
$

The next step is really important. I'll cover in a new blog what happen if you miss this step. For some reasons, gitosis-init doesn't set the right permissions on folders.


git@bob-PC ~
$ chmod 755 /home/git/repositories/gitosis-admin.git/hooks/ -R

git@bob-PC ~
$ chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update

git@bob-PC ~
$

Setting git user's accounts

Because Gitosis use only one account (git), Gitosis need a way to handle permissions. There is a config file for this purpose named : "gitosis-conf" and a folder named "keydir". Each time you want to modify, add or update permissions or change repositories, you will have to make changes in the repository : gitosis-admin. That repository came with Gitosis and it's a git repository, so you have to clone that repository and push back the changes.

For this time only, you will have to specify the complete path where gitosis-admin is.
git clone localhost:/home/git/repositories/gitosis-admin.git


git@bob-PC ~
$ mkdir git_clone

git@bob-PC ~
$ cd git_clone/

git@bob-PC ~/git_clone
$ git clone localhost:/home/git/repositories/gitosis-admin.git
Initialized empty Git repository in /home/git/git_clone/gitosis-admin/.git/
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is 71:3c:3c:96:c6:72:dd:23:3a:dd:3d:b9:3d:4b:08:29.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 0), reused 5 (delta 0)
Receiving objects: 100% (5/5), done.

git@bob-PC ~/git_clone
$

Let's add administrator permissions to our Windows user named bob. There are 3 steps for that.

  • Step 1 : add the user's ssh-rsa public key into gitosis/keydir (file must be named .pub)
  • Step 2 : add the user into the file gitosis.conf
  • Step 3 : commit and push the changes

Step 1 : add ssh-rsa public key

Remember when I said that we will need our bob key. It's for this step.


git@bob-PC ~/git_clone
$ cd gitosis-admin/keydir

git@bob-PC ~/git_clone/gitosis-admin/keydir
$ cp /home/git/../../../cygdrive/c/temp/bob.pub .

git@bob-PC ~/git_clone/gitosis-admin/keydir
$ ls -la
total 2
drwxr-xr-x+ 1 git None   0 2010-09-05 18:15 .
drwxr-xr-x+ 1 git None   0 2010-09-05 18:14 ..
-rw-r--r--  1 git None 392 2010-09-05 18:15 bob.pub
-rw-r--r--  1 git None 392 2010-09-05 18:14 git@bob-PC.pub

git@bob-PC ~/git_clone/gitosis-admin/keydir
$ cd ..

git@bob-PC ~/git_clone/gitosis-admin
$

Step 2 : add user in gitosis.conf


git@bob-PC ~/git_clone/gitosis-admin
$ vim gitosis.conf

You will see that :


[gitosis]

[group gitosis-admin]
writable = gitosis-admin
members = git@bob-PC

That file was generated with gitosis-init command. We will add a line to show debug message when we do git commands. You can remove it later when you will be familiar with Gitosis, but right now it will be useful to debug problems. To add text with Vim, use the letter "i" , enter your text and "espace" to exit insert mode. To save the file, just press "espace" key and enter : wq . That will write and quit.


[gitosis]
loglevel = DEBUG

[group gitosis-admin]
writable = gitosis-admin
members = git@bob-PC

To add Administrator rights for a user, simply add his userid (must be the same as the file put in keydir without the .pub extension) in the section member in [group gitosis-admin].


[gitosis]
loglevel = DEBUG

[group gitosis-admin]
writable = gitosis-admin
members = git@bob-PC bob

Step 3 : commit and push

If you are new to git, I suggest that you read a git tutorial. To export your changes to Gitosis, you have to do these commands :

  • git add .
  • git commit -m 'message'
  • git push

git@bob-PC ~/git_clone/gitosis-admin
$ git add .

git@bob-PC ~/git_clone/gitosis-admin
$ git commit -m 'add user bob'
[master 66cda63] add user bob
 2 files changed, 6 insertions(+), 1 deletions(-)
 create mode 100644 keydir/bob.pub

git@bob-PC ~/git_clone/gitosis-admin
$

git@bob-PC ~/git_clone/gitosis-admin
$ git push
Counting objects: 8, done.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 764 bytes, done.
Total 5 (delta 0), reused 0 (delta 0)
To localhost:/home/git/repositories/gitosis-admin.git
   a1c7b83..66cda63  master -> master

git@bob-PC ~/git_clone/gitosis-admin
$

At this point our Windows user bob should have Administrator rights. You shouldn't have to use the user git to edit gitosis-admin anymore. To be sure that the config was updated in Gitosis, run this command :


git@bob-PC ~
$ cat .gitosis.conf
[gitosis]
loglevel = DEBUG

[group gitosis-admin]
writable = gitosis-admin
members = git@bob-PC bob


git@bob-PC ~
$

Create a new repository

The final test will be to push a new repository into Gitosis. To create a new repository, we need :

  • clone gitosis-admin
  • edit gitosis.conf
  • push modifications
  • create new repository
  • push repository

With your Windows account (not git) you will do the steps.


git@bob-PC ~
$ exit
logout
Connection to localhost closed.

bob@bob-PC ~/sources/gitosis
$ cd

bob@bob-PC ~
$ git clone git@localhost:gitosis-admin.git
Initialized empty Git repository in /home/bob/gitosis-admin/.git/
DEBUG:gitosis.serve.main:Got command "git-upload-pack 'gitosis-admin.git'"
DEBUG:gitosis.access.haveAccess:Access check for 'bob' as 'writable' on 'gitosis
-admin.git'...
DEBUG:gitosis.access.haveAccess:Stripping .git suffix from 'gitosis-admin.git',
new value 'gitosis-admin'
DEBUG:gitosis.group.getMembership:found 'bob' in 'gitosis-admin'
DEBUG:gitosis.access.haveAccess:Access ok for 'bob' as 'writable' on 'gitosis-ad
min'
DEBUG:gitosis.access.haveAccess:Using prefix 'repositories' for 'gitosis-admin'
DEBUG:gitosis.serve.main:Serving git-upload-pack 'repositories/gitosis-admin.git
'
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 10 (delta 0), reused 5 (delta 0)
Receiving objects: 100% (10/10), done.

bob@bob-PC ~
$ git config --global user.name "Bob Ette"

bob@bob-PC ~
$ git config --global user.email bob.ette@ca.ca

bob@bob-PC ~
$

Create our first repository

Let create a new repository named "first_repo".

  • Add a new block [group]
  • Set repository name in property writable
  • Give write permission to users

bob@bob-PC ~
$ cd gitosis-admin/

bob@bob-PC ~/gitosis-admin
$ vim gitosis.conf


[gitosis]
loglevel = DEBUG

[group gitosis-admin]
writable = gitosis-admin
members = git@bob-PC bob

[group first_repo]
writable = first_repo
members = bob


You have to push the changes to Gitosis repository.


bob@bob-PC ~/gitosis-admin
$ git add .

bob@bob-PC ~/gitosis-admin
$ git commit -m 'add repo first_repo'
[master ef13c03] add repo first_repo
 1 files changed, 4 insertions(+), 0 deletions(-)

bob@bob-PC ~/gitosis-admin
$ git push
DEBUG:gitosis.serve.main:Got command "git-receive-pack 'gitosis-admin.git'"
DEBUG:gitosis.access.haveAccess:Access check for 'bob' as 'writable' on 'gitosis
-admin.git'...
DEBUG:gitosis.access.haveAccess:Stripping .git suffix from 'gitosis-admin.git',
new value 'gitosis-admin'
DEBUG:gitosis.group.getMembership:found 'bob' in 'gitosis-admin'
DEBUG:gitosis.access.haveAccess:Access ok for 'bob' as 'writable' on 'gitosis-ad
min'
DEBUG:gitosis.access.haveAccess:Using prefix 'repositories' for 'gitosis-admin'
DEBUG:gitosis.serve.main:Serving git-receive-pack 'repositories/gitosis-admin.gi
t'
Counting objects: 5, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 376 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: INFO:gitosis.run_hook:Running hook post-update
remote: DEBUG:gitosis.gitdaemon:Global default is 'deny'
remote: DEBUG:gitosis.gitdaemon:Walking '.', seeing ['gitosis-admin.git']
remote: DEBUG:gitosis.gitdaemon:Deny 'gitosis-admin'
remote: INFO:gitosis.run_hook:Done.
To git@localhost:gitosis-admin.git
   66cda63..ef13c03  master -> master

bob@bob-PC ~/gitosis-admin
$

At this point Gitosis is ready to receive a new repository. We need to create a local git repository and push it in your server.


bob@bob-PC ~
$ mkdir first_repo

bob@bob-PC ~
$ cd first_repo/

bob@bob-PC ~/first_repo
$ git init
Initialized empty Git repository in /home/bob/first_repo/.git/

bob@bob-PC ~/first_repo
$ touch readme

bob@bob-PC ~/first_repo
$ git add .

bob@bob-PC ~/first_repo
$ git commit -m 'add readme'
[master (root-commit) 09d3a30] add readme
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 readme

bob@bob-PC ~/first_repo
$ git remote add origin git@localhost:first_repo.git

bob@bob-PC ~/first_repo
$ git push origin master
DEBUG:gitosis.serve.main:Got command "git-receive-pack 'first_repo.git'"
DEBUG:gitosis.access.haveAccess:Access check for 'bob' as 'writable' on 'first_r
epo.git'...
DEBUG:gitosis.access.haveAccess:Stripping .git suffix from 'first_repo.git', new
 value 'first_repo'
DEBUG:gitosis.group.getMembership:found 'bob' in 'gitosis-admin'
DEBUG:gitosis.group.getMembership:found 'bob' in 'first_repo'
DEBUG:gitosis.access.haveAccess:Access ok for 'bob' as 'writable' on 'first_repo
'
DEBUG:gitosis.access.haveAccess:Using prefix 'repositories' for 'first_repo'
Initialized empty Git repository in /home/git/repositories/first_repo.git/
DEBUG:gitosis.gitdaemon:Global default is 'deny'
DEBUG:gitosis.gitdaemon:Walking '.', seeing ['first_repo.git', 'gitosis-admin.gi
t']
DEBUG:gitosis.gitdaemon:Deny 'first_repo'
DEBUG:gitosis.gitdaemon:Deny 'gitosis-admin'
DEBUG:gitosis.serve.main:Serving git-receive-pack 'repositories/first_repo.git'
Counting objects: 3, done.
Writing objects: 100% (3/3), 200 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@localhost:first_repo.git
 * [new branch]      master -> master

bob@bob-PC ~/first_repo
$

OH YA ! It's done ! We got our first repository created and added to Gitosis.

In this guide, the client and server are on the same computer. If you used this guide to setup your company repositories, there are few alternatives you could use to setup your clients. You can install cygwin or MSysGit or TortoiseGit. I prefer cygwin because all the tools are included in the same installation.

If you have problems, I suggest to read again this blog, or read this Git Repository Server Gitosis on Win7 : Troubleshooting. about git toubleshooting.

You can follow me on Twitter

Atmosphere : Resin and Websockets

I want to let you know that since the release of Caucho Resin 4.0, Atmosphere applications can be deploy on Resin and will be supported natively. Resin 4.0 implements Servlet 3.0.

That’s really good. Another web server that is supported.

What about the Websocket’s Caucho implementation… nah ! There implementation is too buggy right now. I suppose the code it using a early release of the Websocket specs, and never updated it. Anyway, Websocket application won’t work on Resin 4.0 :(

We posted a thread on there forum here : http://forum.caucho.com/showthread.php?t=10680

but still didn’t received news from them. So until they fix it, I suggest that you try your Websocket application on another web server like Glassfish.

You can follow me on Twitter

Native Websphere support for Atmosphere not yet

Just a little note that we try to add Websphere native support for Atmosphere. We work really hard to find an API that could allow us to do that, but is simply impossible.

We were able to get in touch with the dev team of Websphere and the dev team of WebSphere Application Server Feature Pack for Web 2.0. They have demo that used bayeux for comet application or JMS to simulate asynchronous connections, but it’s not a complete asynchronous solution.

Even with the addon WebSphere Application Server Feature Pack for Web 2.0, you can’t suspend and resume a connection. Websphere server doesn’t support that. It’s not supported in WAS 7.0, but in 8.0 alpha there should be Servlet 3.0 support.

Until IBM do something about that, or wait for WAS 8.0, you will still be able to deploy Atmosphere application on WAS but it will use BlockingIO (one blocking thread by connection).

If that is not enough for you, I suggest Glassfish as alternative. Of course, we will continue to monitor the situation. Maybe the community call could be enough to wakeup them :)

You can follow me on Twitter

Grizzly Deployer was adopted (almost) by Akka

Just to let you know that Grizzly Deployer was adopted (almost) by another framework : Akka.

Since May 22, 2010, viktorklang is working in a branch : deployer. here the description in the commit “Starting to add Deployer [viktorklang]”

You can browse the code on github : http://github.com/jboner/akka/tree/deployer/akka-core/

I hope to see that included in a next release.

You can follow me on Twitter

Grizzly Deployer was adopted by Atmosphere

Just to let you know that Grizzly Deployer was adopted by another framework : Atmosphere.

Since Atmosphere-0.6, Grizzly Deployer is in the module : spade-server

To launch Atmosphere application you could use Grizzly Deployer or use Atmosphere-Spade-Server (that used Grizzly Deployer under the hood).

You can browse the changes on github : http://github.com/Atmosphere/atmosphere/tree/master/extras/spade-server

I did the commit under the name “Unknown” :) It was my first git commit.

You can follow me on Twitter

Grizzly Deployer (1.9.19) new features

It has been a while since a blog about Grizzly, I was too busy adding new features. The Grizzly Deployer’s community grows and requested theses new features.

Here a quick list of the changes :

  • Websockets support
  • Watch folder
  • Can starts without applications to deploy
  • Can be embedded and extended easily
  • and few bug fixes (thanks to the community)

A little overview or remainder of what is Grizzly Deployer

Grizzly Deployer is a bundle of Grizzly project that act as a web container. It will allow you to deploy applications (war, servlet).
Works fine for Comet, Atmosphere, JSP applications … For more details, I suggest this link Introduction to the first release of Grizzly Deployer

First, lets take a look at the command line parameters.


Usage: com.sun.grizzly.http.servlet.deployer.GrizzlyWebServerDeployer

  -a, --application=[path]    Application(s) path(s).

                              Application(s) deployed can be :
                              Servlet(s), war(s) and expanded war folder(s).
                              To deploy multiple applications
                              use File.pathSeparator

                              Example : -a /app.war:/servlet/web.xml:/warfolder/

  -p, --port=[port]           Runs Servlet on the specified port.
                              Default: 8080

  -c, --context=[context]     Force the context for a servlet.
                              Only valid for servlet deployed using
                              -a [path]/[filename].xml

  --dontstart=[true/false]    Won't start the server.
                              You will need to call the start method.
                              Useful for Unit testing.
                              Default : false

  --libraryPath=[path]        Add a libraries folder to the classpath.
                              You can append multiple folders using
                              File.pathSeparator

                              Example : --libraryPath=/libs:/common_libs

  --autodeploy=[path]         AutoDeploy to each applications.
                              You could add JSP support.
                              Just add a web.xml that contains Jasper

                              Example : --autodeploy=/autodeploy

  --webdefault=[path]         webdefault to be used by all applications, can be file or dir with multipe web.xmls.
                              If you want to add only one webdefault point it to web.xml file,
                              If you want multiple files to be included put them in one dir and provide this location here.

                              Example : --webdefault=webdefault.xml

  --cometEnabled=[true/false] Starts the AsyncFilter for Comet.
                              You need to active this for comet applications.
                              Default : false

  --websocketsEnabled=[true/false] Starts the AsyncFilter for Websockets.
                              You need to active this for websockets applications.
                              Default : false

  --forceWar=[true/false]     Force war's deployment over a expanded folder.
                              Will deploy the war instead of the folder.
                              Default : false

  --ajpEnabled=[true/false]   Enable mod_jk.
                              Default : false

  --watchInterval=[seconds]   Watch interval to scan for new applications to deploy in work folder.
                              Default : -1 ; disabled

  --watchFolder=[path]        Folder to scan for new applications to deploy in work folder
                              Default : none

Websocket support

The first feature added was Websockets support. It can be enabled at command line by using the parameter –websocketEnabled=true . That will allow you to deploy Websocket applications.

Watch folder

This feature is interesting, because most of the web container have it. Grizzly Deployer will watch a specific folder to applications to deploy. When this feature was added, I remove the need to specify a application to deploy by command line.
To enable this feature you need to use the command line parameter –watchFolder=[path]. It can be used with this other parameter : –watchInterval=[seconds].

The parameter –watchInterval is used to create a watchdog that will monitor the watch folder. If the parameter –watchInterval is not present, when Grizzly Deployer starts, it will check
all applications that are in the watch folder at launch time and will deploy them, but it won’t monitor the folder after that.

When both of theses parameters are used, it will create a watchdog that will deploy and undeploy application put in the watch folder, and remove from that folder. If you deploy an application at command line and put another application with the
same context name in the watch folder, the application in the watch folder won’t be deploy twice.

Embedded Grizzly Deployer

Grizzly Deployer could already be embedded trough his API, but we got few feedback about to make it simplier. The problem wasn’t the API, it just that most people will only use the main part of Grizzly Deployer : deploy a war.

So to make it easier and faster to implement, I did a little refactoring. We now have 2 main config classes we could use to launch an Embedded Grizzly Deployer : DeployerServerConfiguration and DeployableConfiguration.

DeployerServerConfiguration

That file is used to launch Grizzly Deployer. It contains the same command line parameters. In fact, at runtime, the command line parser use that classe.


    GrizzlyWebServerDeployer deployer = new GrizzlyWebServerDeployer();
    
    DeployerServerConfiguration conf = new DeployerServerConfiguration();
    conf.cometEnabled = true;
    conf.watchFolder="C:/temp/webapps/";
    conf.watchInterval=15;
    deployer.launch(conf);                             

Once grizzly Deployer is launched in your application, you can deploy a war later. For that, you use this class : DeployableConfiguration.


    DeployableConfiguration warConfig = new DeployableConfiguration();  
    warConfig.location = "C:/temp/webapps/grizzly-jmaki.war";
    
    deployer.deployApplication(warConfig);                           

You can follow me on Twitter.

Serialization surprises

CONTEXT

I want to share a problem that we had in our project. We were doing a real-time “Profits and Loss” server (P&L). The server sends stock updates to all the users subscribed to the stocks, basically as Google Finance or Yahoo Finance.

SIMPLE IMPLEMENTATION

I will used a basic approach (no aggregation and no optimization) to explain the problem that we had with Serialization and Hessian.

The server always keep the last value on the stocks, because when a user ask for a quote we want to send back as soon as possible the
last value (from the cache) that we had on that stock).

We kept the stock prices (BID and ASK) in a simple object : SymbolSerializable.

When a update is received for a stock, we update the values in his SymbolSerializable and send back the updated values to all
client subscribed to this stock.


...

symbol.setBid(update.getBid());
symbol.setAsk(update.getAsk());

...

for(Client client : clientList){
    client.sendUpdate(symbol);
}

....

It can’t be more simpler than that. The client will received each updates on his stocks.

The problems that we had was surprising. The clients were receiving the always the same price on a stock !

WHY ?

here an example :

stock : ABC

Updates :

1 : bid=10.25$, ask=10.50$
2 : bid=11.50$, ask=11.75$
3 : bid=12.00$, ask=12.15$

and the clients received :

1 : bid=10.25$, ask=10.50$
2 : bid=10.25$, ask=10.50$
3 : bid=10.25$, ask=10.50$

To debug our server we printed the values sent to the client, and we saw in the server’s logs that the values were corrects.

value sent to client :
bid=10.25$, ask=10.50$

value sent to client :
bid=11.50$, ask=11.75$

value sent to client :
bid=12.00$, ask=12.15$

Everything look fine, why it doesn’t work on the client side ?

INVESTIGATION/SOLUTION

The only thing that could cause the problem were our Serialization in the server.

I’ll show you four different test cases. See the implementations below.

  • #1 : Reference implementation using Serializable (failed)
  • #2 : Changed Serializable for Externalizable interface (failed)
  • #3 : Serializable using new (passed)
  • #4 : Serializable using reset (passed)
#1 : Serializable : Reference #2 : Externalizable #3 : Serializable with new #4 : Serializable with reset
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • All items read
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • All items read
  • bid= 0.0 ask= 0.0
  • bid= 1.0 ask= 1.0
  • bid= 2.0 ask= 2.0
  • bid= 3.0 ask= 3.0
  • bid= 4.0 ask= 4.0
  • bid= 5.0 ask= 5.0
  • bid= 6.0 ask= 6.0
  • bid= 7.0 ask= 7.0
  • bid= 8.0 ask= 8.0
  • bid= 9.0 ask= 9.0
  • All items read
  • bid= 0.0 ask= 0.0
  • bid= 1.0 ask= 1.0
  • bid= 2.0 ask= 2.0
  • bid= 3.0 ask= 3.0
  • bid= 4.0 ask= 4.0
  • bid= 5.0 ask= 5.0
  • bid= 6.0 ask= 6.0
  • bid= 7.0 ask= 7.0
  • bid= 8.0 ask= 8.0
  • bid= 9.0 ask= 9.0
  • All items read

Here the Serializable pojo


package ca.sebastiendionne.demo.model;

import java.io.Serializable;

public class SymbolSerializable implements Serializable {

    private static final long serialVersionUID = 7853892880704628717L;

    Double bid = null;
    Double ask = null;

    public SymbolSerializable(){
    }

    public Double getBid() {
        return bid;
    }
    public void setBid(Double bid) {
        this.bid = bid;
    }
    public Double getAsk() {
        return ask;
    }
    public void setAsk(Double ask) {
        this.ask = ask;
    }

    @Override
    public String toString() {
        return "bid = " + bid + "  ask=" + ask;
    }

}

Here the Externalizable pojo


package ca.sebastiendionne.demo.model;

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;

public class SymbolExternalizable implements Externalizable {
    private static final long serialVersionUID = 1853892880704628717L;

    Double bid = null;
    Double ask = null;
    
    public SymbolExternalizable(){
        
    }
    
    public Double getBid() {
        return bid;
    }
    public void setBid(Double bid) {
        this.bid = bid;
    }
    public Double getAsk() {
        return ask;
    }
    public void setAsk(Double ask) {
        this.ask = ask;
    }
    
    @Override
    public String toString() {
        return "bid = " + bid + "  ask=" + ask;
    }

    @Override
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        bid = (Double)in.readObject();
        ask = (Double)in.readObject();
    }

    @Override
    public void writeExternal(ObjectOutput out) throws IOException {
        out.writeObject(bid);
        out.writeObject(ask);
    }
    
}

For the test cases I use a for loop to create Symbol and serialized on the hard drive and unserialized them back.

Here the code for the test case #1 (used as reference)


package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolSerializable;

public class SerializationFailed {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolSerializable symbol = new SymbolSerializable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            oos.writeObject(symbol);
            oos.flush();
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolSerializable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

Here the code for the test case #2


package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolExternalizable;

public class SerializationFailed2 {

    
    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolExternalizable symbol = new SymbolExternalizable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            oos.writeObject(symbol);
            oos.flush();
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolExternalizable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

Here the code for the test case #3


package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolSerializable;

public class SerializationWithNew {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolSerializable symbol = new SymbolSerializable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            SymbolSerializable symbol2 = new SymbolSerializable();
            symbol2.setAsk(symbol.getAsk());
            symbol2.setBid(symbol.getBid());
            
            oos.writeObject(symbol2);
            oos.flush();
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolSerializable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

Here the code for the test case #4


package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolSerializable;

public class SerializationWithReset {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolSerializable symbol = new SymbolSerializable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            oos.writeObject(symbol);
            oos.flush();
            oos.reset(); // magic line
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolSerializable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

I can't say what is the performance impact if we use reset() on each updates sent. It is better to use new or reset ?

You can follow me on Twitter : http://twitter.com/survivant

Enhance your javadoc with ULMGraph

We can’t live without Javadoc, but even if it useful, it’s not complete. One missing thing is UML within the Javadoc.

To add UML to your Javadoc, is quite simple. You need to add Graphviz into your maven build.

First you need to download and install Graphviz. Go there Graphviz

After that you should add the variable GRAPHVIZ_HOME (that point to the installation folder) into your system.

The last step is to add the plugin into your pom.

Add the lines in bold.


<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-javadoc-plugin</artifactId>
  <version>2.6.1</version>
  <configuration>
    <links>
      <link>http://java.sun.com/javase/6/docs/api/</link>
    </links>
    <detectOfflineLinks />
    <doclet>org.umlgraph.doclet.UmlGraphDoc</doclet>
    <!--
      use this line or use the variable GRAPHVIZ_HOME
      <docletPath>/path/to/UmlGraph.jar</docletPath>
    -->
    <docletArtifact>
      <groupId>org.umlgraph</groupId>
      <artifactId>doclet</artifactId>
      <version>5.1</version>
    </docletArtifact>

    <additionalparam>-operations</additionalparam>
    <additionalparam>-qualify</additionalparam>
    <additionalparam>-types</additionalparam>
    <additionalparam>-visibility</additionalparam>
    <additionalparam>-collpackages</additionalparam>
    <show>private</show>
  </configuration>
</plugin>

With that you will have nice UML into your javadoc. I hope that help.

A simplier way is to add that instead in the pom.xml


<plugin>
  <artifactId>maven-javadoc-plugin</artifactId>
  <configuration>
              <show>package</show>
              <version>true</version>
              <javadocVersion>${source.version}</javadocVersion>
              <links>
                  <link>http://java.sun.com/javase/6/docs/api/</link>
                  <link>http://java.sun.com/products/servlet/2.5/docs/servlet-2_5-mr2/</link>
                  <link>http://joda-time.sourceforge.net/api-release/</link>
                  <link>http://findbugs.sourceforge.net/api/</link>
                  <link>http://tomcat.apache.org/tomcat-6.0-doc/api/</link>
                  <link>http://jetty.mortbay.org/jetty/jetty-6/apidocs/</link>
                  <link>http://download.eclipse.org/jetty/stable-7/apidocs/</link>
                  <link>https://grizzly.dev.java.net/nonav/apidocs/</link>
              </links>          <source>1.6</source>
     <code>javadoc:aggregate</code>
      <code>javadoc:test-aggregate</code>
    <doclet>gr.spinellis.umlgraph.doclet.UmlGraphDoc</doclet>
    <docletArtifact>
      <groupId>gr.spinellis</groupId>
      <artifactId>UmlGraph</artifactId>
      <version>4.6</version>
    </docletArtifact>
    <additionalparam>
      -inferrel -inferdep -quiet -hide java.*
      -collpackages java.util.* -qualify
      -postfixpackage -nodefontsize 9
      -nodefontpackagesize 7
    </additionalparam>
  </configuration>
</plugin>