DGit: GitHub’s distributed, sw-defined storage

DGit

GitHub announced a new redundancy-based technology built on core git version control techniques for a more reliable, highly available and performance oriented storage to repositories – DGit. While git is distributed by design (any copy of a repository contains the entire history), it doesn’t support mirroring by itself. DGit stores the data of each repo in 3 different servers. Continue reading DGit: GitHub’s distributed, sw-defined storage

gitfs: version controlled filesystem

You have a server and you want to rollback the directory with your website files anytime you want. You can keep a backup of the files on a separate media and copy them back. What if you made some changes to the theme and want to keep both the original version and the modified one? Redundancy increases and it’s too difficult to trackback. Continue reading gitfs: version controlled filesystem

Move subdirectory to new git repo (preserve history)

git_logo_compI was about to branch a subdirectory within one of my GitHub projects as a new project. I realized that I’ll lose all revision history which I wanted to avoid for obvious reasons. I found a discussion thread on Stack Overflow to achieve this. Here’s the simplified procedure with explanation for each step. Continue reading Move subdirectory to new git repo (preserve history)

Git commands for everyday use

git_logo_compGit is a very advanced and powerful version control system. It supports too many features and it’s difficult to remember each and every command unless you are a heavy Git user or a Git dev yourself. I try not to remember too many of them. However, Git is used best when it is used from the cmdline. And in my experience, there are a few commands that are used much more frequently than others. I will list those in this article to have a reference for myself as well as for others. In the examples I will use GitHub’s own Atom project repository link for the examples.

Before I begin, here’s a good practice for a Git dev – always have a clean master branch checked-out from the original project’s master branch. Make your changes to a separate development branch and rebase your master and dev branch regularly with the main project branch. In other words, your changes should go into the project’s master branch from your dev branch. Those changes are then pulled into your master branch.

Git installation and initial setup
  • Install Git on Ubuntu
    $ sudo apt-get install git
  • Set your name, email address and default editor
    $ git config --global user.name "My Name"
    $ git config --global user.email "your-email@provider.com"
    $ git config --global core.editor vim
Add a project
  • Use GitHub’s web interface to fork your own master branch from the project’s main branch. Clone your mater branch locally.
    $ mkdir atom
    $ cd atom
    $ git clone https://github.com/username/atom.git
  • In case of very old or large repositories, you can do a shallow clone by fetching history of only latest n commits.
    $ git clone --depth 2 https://github/username/atom.git
  • Add the original Atom project as upstream.
    $ git remote add upstream https://github.com/atom/atom.git
  • Use the GitHub web interface to create a new branch named dev. This is where your own changes should go first.
  • Check-out (or switch to) the dev branch
    $ git checkout -b dev
The powerful checkout command
  • Check-out (switch to) a branch, tag, revision or revert changes to a file
    $ git checkout branch/tag/revision/filename
  • Revert all local changes
    $ git checkout -- .
Get upstream changes to your master branch
  • Switch to the master branch
    $ git checkout master
  • Check details of origin/upstream
    $ git remote show origin
    $ git remote show upstream
  • Retrieve the latest changes from upstream
    $ git fetch upstream
  • Merge upstream into your master branch
    $ git merge upstream/master
Get changes from your master to dev branch

A few words of merge and rebase first. Rule of the thumb is – use merge/rebase when you have no changes in your local dev branch. Both result in same commit history. However, if you have changes in your local dev branch but would like to get the changes in remote as well, then use rebase for a clean commit history.

  • Checkout dev branch
    $ git checkout dev
  • Merge or rebase depending on the situation
    $ git merge master
    $ git rebase master
  • Push the changes to your dev branch origin
    $ git push origin dev
Branch deletion
  • Delete a local branch
    $ git branch -D branch_name
  • Push the deletion to remote
    $ git push origin --delete branch_name
Other common scenarios
  • Check the branch and local status (maybe after a merge/rebase conflict)
    $ git status
  • Check and local modifications to tracked files
    $ git diff
  • Set diffuse as the diff tool
    $ git config --global diff.tool vimdiff
    $ git config --global difftool.prompt false
    $ git config --global alias.d difftool

    To view the diff in diffuse

    $ git d
    $ git d commit#1 commit#2
    $ git diff HEAD^ HEAD
  • Check commit history
    $ git log --stat
  • Hide local changes and pull, then unhide local changes
    $ git stash
    $ git pull
    $ git stash pop
  • Commit changes locally
    $ git commit -a -s
  • Push the changes to origin
    $ git push origin dev
  • Undo a commit, make changes, redo it
    $ git commit ...
    $ git reset --soft HEAD^
    //edit files
    $ git add ...
    $ git commit -c ORIG_HEAD
  • Edit/amend the most recent commit message
    $ git commit --amend -m "New commit message"

    If the commit is already pushed to remote

    $ git push origin dev --force
  • Clean local changes
    $ git reset --hard HEAD
    $ git clean -f -d //cleans all untracked files as well
  • Reset unmerged files
    $ git reset filename
    $ git checkout filename
  • Accept all changes post merge conflict occurs
    $ git reset --hard origin/dev
    $ git merge -X theirs master
  • If you have lost the URL of the remote repo you forked
    // if referential integrity has been broken:
    $ git config --get remote.origin.url
    // if referential integrity is intact:
    $ git remote show origin
Tagging

Tags are baselines you can revert back to in the future.

  • Get and list tags
    $ git pull --tags
    $ git tag
  • Create a tag
    $ git tag -a tag_name -m "your message here"
    $ git push origin --tags
  • Delete a tag
    $ git tag -d tag_name
    $ git push origin :refs/tags/tag_name

Another useful list is the giteveryday man page.

Feel free to suggest if you think I should add any command to the article.

FreeFileSync: synchronize files and directories

I am a little paranoid about losing my data. I keep multiple backups of lesser sensitive data in different cloud storage services. For sensitive data the strategy is to store them mirrored in external storage. However, it is impossible to keep track of thousands of files on different disks. Once in a while I use FreeFileSync to help me detect any difference in data on my external disks. FreeFileSync is a very powerful open source utility with an array of industry standard features:

  • Detect moved and renamed files and folders
  • Copy locked files (Volume Shadow Copy Service)
  • Detect conflicts and propagate deletions
  • Binary file comparison
  • Configure handling of Symbolic Links
  • Automate sync as a batch job
  • Process multiple folder pairs
  • Comprehensive and detailed error reporting
  • Copy NTFS extended attributes (compressed, encrypted, sparse)
  • Copy NTFS security permissions
  • Support long file paths with more than 260 characters
  • Fail-safe file copy
  • Expand environment variables like %UserProfile%
  • Access variable drive letters by volume name (USB sticks)
  • Native 64-bit support
  • Keep versions of deleted/updated files
  • Prevent disc space bottlenecks via optimal sync sequence
  • Full Unicode support
  • Highly optimized runtime performance
  • Include/exclude files via filter
  • FreeFileSync portable and local installation available
  • Handle daylight saving time changes on FAT/FAT32
  • Use macros %time%, %date%, et al. for recurring backups
  • Case-sensitive synchronization
  • Built-in locking: serialize multiple jobs running against the same network share
  • Multiplatform: Linux, Windows, Mac OS X

Installation

To install FreeFileSync on Ubuntu, download the packages from the downloads section.

Webpage: FreeFileSync

ungit: the easiest way to Git!

[youtube=https://www.youtube.com/watch?v=hkBVAi3oKvo]

Git is a very advanced version control system and using Git can be tricky if you are new to it. ungit is a great aid in spending less time in learning Git than in actual development. It is a web-based application that uses node.js. IMHO it is also a very helpful utility to learn Git visually first than jumping into the cmdline with Git. The video above gives a quick overview of how ungit can help you stay off the cmdline to use Git. The visual representations of the probable actions are simply brilliant!

ungit is multiplatform and tightly integrated with GitHub. It has a plugin based architecture so it can be extended easily to support other source control services those support Git.

You need node.js, npm (comes with node.js) and git to use ungit. Necessary instructions to use ungit are given in the ungit GitHub page (link below).

Webpage: ungit

QGit & Diffuse: Git GUI on Linux

qgit_compI was looking for a good GUI based Git repository browser to view changes. To speak the truth, the terminal is not really the best option to look at a overwhelming number of change records which need to be examined in detail and in reasonable time. There are many Git GUI interfaces on Linux but some of them are bulky and some lack finesse. I found QGit to be exactly the tool I needed.

QGit features

  • Very lightweight. Written using Qt/C++.
  • Auto branch configuration detection
  • Shows everything you need to understand a change: commit history (including range), change log for individual commit, diff, modified file list
  • Can launch an external diff viewer like Diffuse
  • File tree view that helps to track commits to a single file
  • Save a patch series to send for review, apply a patch at any revision, view a patch
  • User defined keyboard shortcut for Actions
  • Commit, amend commit, make branch, make tag
  • Available by default in the repositories for major distros

QGit integrates seamlessly with Diffuse, a graphical file diff viewer. Diffuse is one of the few diff-merge tools that supports N-ary file merge. However, one of its most powerful features is the support for version control systems. Diffuse is written in Python.

Diffuse

Diffuse features

  • Merge N number of files side by side in different panes
  • Manually adjust line matching and directly edit files
  • Integrates with file manager menu
  • Modify both left and right side files (2-way merge)
  • Retrieve revisions of files from Bazaar, CVS, Darcs, Git, Mercurial, Monotone, SVN and SVK repositories for comparison and merging

Installation

To install QGit and Diffuse on Ubuntu, run:

$ sudo apt-get install qgit diffuse

Webpage: QGit, Diffuse

tig

Tig is an ncurses-based text-mode interface for git. It functions mainly as a Git repository browser, but can also assist in staging changes for commit at chunk level and act as a pager for output from various Git commands. To browse a git repository navigate inside a versioned directory and run tig. To install tig on Ubuntu:

$ sudo apt-get install tig

SmartGit/Hg: cool GUI client for Git and Mercurial

smartgitIf you are related to Linux development, you probably have heard of Git already. It is a powerful and widely popular version control system. The Linux kernel project is managed using Git. Git does not have any GUI of its own and needs to be accessed from the cmdline. SmartGit/Hg is a multi-platform (Linux, Windows, Mac) GUI client for Git and Mercurial (Hg). It is simple to use, has powerful features and comes as a bundle that packages utilities like SSH, FTP etc. so that you do not need to install anything separately. As you can see from the above image, the interface is pretty attractive as well. SmartGit/Hg depends on the JAVA runtime environment.

Webpage: SmartGit/Hg

Fossil: your local version control system

fossil_compWe keep changing documents and files and it may not be possible to track all the changes or go back to a previous state after a few days. That’s where a revision control system comes handy. While there are many cloud based solutions it is difficult to find something which runs locally and yet not too heavy. Enter Fossil – multiplatform, damn light (the Linux package is less than 1MB in size!!!) and lots of functionality and options for customization along with a web interface. You can change the settings to view diffs with your favourite diff viewer. Fossil is written in C. The official documentation is extensive. Here’s a quick guide to kickstart with Fossil:

$ mkdir /opt/myrepo //your main repository location
$ cd /opt/myrepo
$ fossil init localrepo //create a repository localrepo
$ fossil set editor vim //let vim be your editor for changelogs
$ mkdir ~/fossil_files //we'll check out here
$ cd ~/fossil_files
$ fossil open /opt/myrepo/localrepo
$ mkdir root
$ touch root/firstfile //your first file, add something in it
$ fossil commit //you just added something in your repo
$ fossil ui /opt/myrepo/localrepo //the web interface to browse repos, branches, files...

Click on the Files tab and now you can browse your newly created repository, view diffs, changelogs and everything. How cool is that?

To install Fossil on Ubuntu:

$ sudo apt-get install fossil

Navigate through the documentation to know more about Fossil. You’ll be surprised what you can do with it.

Chisel is a service offering hosting for unlimited Fossil repositories.

Webpage: Fossil