From knok at daionet.gr.jp Mon Jul 12 17:28:38 2004 From: knok at daionet.gr.jp (knok@daionet.gr.jp) Date: Mon Jul 12 19:23:33 2004 Subject: [Namazu-devel-en] CVS repository of namazu module was returned. Message-ID: <87y8lpk4sp.wl@knok.daionet.gr.jp> CVS repository of namazu module was returned. We decided to return the repository because we verified various methods about it; compared the repository and rsynced backup, HEAD and working copy, release files and release point tags. Then we couldn't find any unauthorized modification. We can't guarantee the repository was not infected, but we think that has no problem to develop from HEAD and latest branches. You need a notice to refer old source code, but we took priority over continuous development. Please understand our decision. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.namazu.org/pipermail/namazu-devel-en/attachments/20040712/dc7fb239/attachment.bin From knok at daionet.gr.jp Fri Jul 23 15:03:44 2004 From: knok at daionet.gr.jp (knok@daionet.gr.jp) Date: Fri Jul 23 15:03:47 2004 Subject: [Namazu-devel-en] karin.namazu.org compromise report Message-ID: <87d62nqmyn.wl@knok.daionet.gr.jp> karin.namazu.org compromise report We reports details of karin.namazu.org compromise. karin.namazu.org was the main server of Namazu Project. Time Table (JST) 5/19 Debian Project released new cvs package to fix security issue. 5/23 02:24 Occurred the first intrusion via CVS. The intruder sent some monitoring tools like ttymon. 05:03 He sent some other tools a kind of keylogger. 18:04-18:40 He created "test" user, and copied passwd and shadow files into his home directory. 18:42 Exploited passwd and shadow file via ftp. 19:07 The intruder was logined as the user "jitterbug" via ssh. 19:56 Occurred a CVS access to escalate root privilege from jitterbug. The detail was unknown. 19:58 He installed a kind of rootkit. 5/24 19:00 Network in the subnet of the place of karin.namazu.org was bursted. 20:00 We find the compromise. 5/25 02:00 karin.namazu.org was unplugged from the network. See http://www.namazu.org/#restoration for services restoration info. Machines and services at the incident CVS pserver was served as root privilege via inetd on karin. The CVS server provided many software and so many users can access via pserver or ssh. Anonymous user also can get sources via pserver. karin was build on Debian GNU/Linux 3.0. Latest cvs package was released in 5/19, but we didn't upgrade it while the incident. Then karin was unplugged by switching hub, and their HDD was moved into another machine and analyzed. In the result, we found some rootkit in the HDD. We decided 5/23 02:24 JST is the first intrusion time because some rootkit files and CVS temporary directory had same ctime. CVS pserver had root privilege, so the intruder could get root privilege easily. And jitterbug account had temporary password to set spam filter, and it was not removed, so he can get the password from shadow file easily. Recovering services karin was old hardware, so we have a plan to move newer network and machine named "vaj.namazu.org", and it was already placed. So we moved all services into vaj.namazu.org. Inspection CVS repository karin had two HDDs, and one of them was used for backup. The original CVS repository was daily backuped by rsync. The original repository was daily accessed to make ChangeLog graph, so almost files in the repository had same atime. On the other hand, rsynced backup repository files had correct atime exclude directories. We considered the reason is that rsync accessed only directory, then if any file was not changed, rsync didn't touch the file. It is possible to modify a file with keeping atime, but it requires to record the atime before modify. We considered the possibility of modify repository such complex sequences without inconsistency is very low. Then we checked difference between the backup and original, and we can find only correct updates. And we checked the release points version 2.0.12 and 2.0.13 from PGP signed archives, and we can't find any difference. Furthermore we checked the stable branch. A developer has a working copy at 5/13, so he checked 5/13 stable branch and it and couldn't find any difference, and checked further commits was correct. We also checked HEAD trunk with same method, and it seems no problem. In the result, we considered the CVS repository is almost safe, and continue to use it. Further operations Now we operate the following policies: - CVS pserver runs in chroot environment with non-root privilege, and it has the copy from original repository. It is for anonymous access only. -- The environment is built by Debian cvsd package. It is easy to update cvs command. - Reinforcement administration team. There is a mailing list for admin team, and the ml subscribe debian-security-announce list. - Non-admin members are only access CVS via ssh. It is a new accounting policy. - Make some backups from another network machines. We are trying to operate more safety with the experience. The analysis was cooperated with NetVillage Co., Ltd. Jul 23, 2004 NOKUBI Takatsugu -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://www.namazu.org/pipermail/namazu-devel-en/attachments/20040723/cf4444cc/attachment.bin