Thursday, 20 September 2012

Bash history of unique commands

Ctrl-r in the bash is a great feature. I use it like bookmarks in a webbrowser. But you loose commands very fast, they are pushed out of the history. There are many native ways (HISTCONTROL) to make bash history entries unique. None worked for me, native bash settings either work for joining multiple bash sessions or enforcing uniqueness, both features at once doesn't work seamless.

I hacked a small python script that cleans the bash history the way I like it.
  • If it removes duplicated entries it will remove the oldest
  • It will only change the history when you enter a new command
  • It will notify you with a + sign that it added a command
  • It will only write to the harddisk if you added a new command
  • It will not loose commands like using HISTCONTROL when joining multple bash sessions history
  • It locks the history file, to avoid race-conditions (usually you don't enter commands in two windows at once, but you never know)
I use it for 2 month by now and I use the bash everyday, every few minutes while working at the moment I have 2665 unique commands:
Sherlock:~ ganwell$ wc -l .my_history 
    2665 .my_history

Changes to .profile:
profile.sh
The source of unique_history:
unique_history.py

6 comments:

  1. I have been thinking lately that command history should be accessible in a similar manner to versioning history:

    * it should be possible to tag certain commands (or command templates) .and access them by tag. These can be implemented simply as comments, with some special persistence.
    * history should branch, depending on what task you're working on; a shell could be assigned to track as a particular branch name, and you may later choose to `base` your current shell on a chosen branch.

    ReplyDelete
    Replies

    1. Hi Joel

      The first part is definitely possible:

      * Unique_history.py matches commands against regex-templates and add a #tag (comment) to the command automatically.
      * Add tags manually "echo hello #test-commands"
      -> Search with ctrl-r "#test...."

      I don't get the second part, I use the unique_history.py, because I usually can only remember part of the commands I used, for example when I last connected to foo.ch I had this port forwarding. So I do ctrl-r, foo.ch, ctrl-r, ctrl-r and eventually I get:

      ssh -L8888:notebookhost.local:8888 foo.ch

      Delete
    2. Second part: I want a different history for the different contexts in which I use a shell. Perhaps that means attaching a different bash history to each virtualenv.

      Delete
    3. Well, you could use environment variable HISTFILE to tell history-script and bash to add commands to a different file.

      Delete
  2. try HISTCONTROL=ignoredups

    as written in bash manpage:

    HISTCONTROL
    A colon-separated list of values controlling how commands are
    saved on the history list. If the list of values includes
    ignorespace, lines which begin with a space character are not
    saved in the history list. A value of ignoredups causes lines
    matching the previous history entry to not be saved. A value of
    ignoreboth is shorthand for ignorespace and ignoredups. A value
    of erasedups causes all previous lines matching the current line
    to be removed from the history list before that line is saved.
    Any value not in the above list is ignored. If HISTCONTROL is
    unset, or does not include a valid value, all lines read by the
    shell parser are saved on the history list, subject to the value
    of HISTIGNORE. The second and subsequent lines of a multi-line
    compound command are not tested, and are added to the history
    regardless of the value of HISTCONTROL.

    ReplyDelete
    Replies
    1. Thanks a lot for your hint. I updated my post to make it clear that:

      Unfortunately I these HISTCONTROL settings don't work seamless if you want to join history of multiple bush-sesssions. I tried all combinations of these settings. You can either have the multi-bash-history or the uniqueness, but not both.

      Delete