Archive for February, 2010

Emacs Package Management

Monday, February 22nd, 2010

One of the thing’s I’ve been thinking about recently is package management for my emacs customizations. I have a large number of emacs packages that are at least intrinsically external to my own repository - 42, I believe - and I have to install Emacs on a new system roughly once every four months. When I recently switched from SVN to HG for hosting my repository, this prompted me to think a lot about how to organize it, and this problem in particular.

Unfortunately, this seems to be a nontrivial problem. The easiest solution would be for all of these packages to be added to emacs core; this is rather unfortunately not going to happen. The three main possible solutions I thought of are the following:

The first option, which I did for a long time, is to just check each package into your emacs customizations repository. This overall is a decent solution; it’s probably what most people use. It has the advantage of making your repository stand-alone - nothing but it has to be checked out. However, it makes it difficult to detect updates to packages you use and to apply those updates. This tends to end up with you having fewer features and more bugs than you otherwise could have, which lead me in search of other solutions.

Option #2 - Hand-rolled package management

The solution I currently have in-place is a set of utilities that will download the latest version of a library if it exists. While this does mostly solve the problem of difficult updates(at the time of a fresh Emacs install, I am fully up-to-date, and can juts use native VC to update further), it introduces a new set of problems. For one, it makes the repository not be packaged together, making it harder to distribute to others. It also requires installing a lot of different version control systems - I think the various modes I need to check out use six different ones. Finally, there are cases when I don’t want the latest version, or need to apply a patch, because it is broken.

The only one of these problems I currently solve is the last one. My repository contains a directory with modes that can’t be managed by the utilities I described above. This can be because they are hard to download pragmatically or the latest version is broken - clojure-mode is an example of the latter, member-functions.el the former. This is essentially Option #1 on a slightly smaller scale.

Option #3 - ELPA

What I’d really like is a finished ELPA, included in a base Emacs distribution. ELPA is the Emacs Lisp Package Archive, a package manager for Emacs, slated for addition to Emacs 24. I’ve tried it, and while it’s a good idea, right now it is not good enough to use - hopefully, this can be fixed, because it seems like this would be a solution to most of my issues. However, right now, there’s the following reasons I don’t currently use it:

  1. Ease of upload - This only seems like it should be an issue for package developers, and thus not affect my use of it in my repository, but I think it’s one reason for later items. Essentially, if it’s hard to upload a new package to ELPA, more package maintainers will just not bother doing so. Right now, the process seems to be to email the package to the ELPA maintainer, who will put it in or update the package, which is clearly not scalable or particularly easy. Ideally, I would be able to register some version control location with ELPA, and have it pick up any changes I make to it.
  2. Requirements for packages - The packages included must meet some irritating requirements, with comments in specific formats and other issues. These aren’t too bad - ELPA needs some way of finding dependencies and requirements - but it also doesn’t support multi-file packages very well. This is a fairly big issue, since without this it can only be used for smaller modes.
  3. Availability of packages - Right now, fewer than 10% of the packages I use are in ELPA. This is a combination of reasons, such as some packages being old, some not meeting the requirements described below, or the developer is just lazy. Whatever the reason, I’d like it to contain more of the packages I personally use, although I’d be willing to help convince others to put their packages on ELPA if my other problems were solved.
  4. Old package versions - Some of the package versions on ELPA, such as eproject, are quite old. I suspect this relates to issue #1 above.
  5. No update/revert mechanism - Once you’ve installed a package, there’s no way to update it if ELPA receives an update. There also isn’t a way to revert to a previous deployment.

For now, I don’t really have a good solution. Hopefully, ELPA will improve - I may work on that some myself - but in the meantime, if you have your own system for managing external packages, please let me know in the comments.

Emacs Minor-Modes: mic-paren, pager, dired-isearch, whichfunc, winpoint, and highlight-parentheses

Monday, February 15th, 2010

I haven’t written about recent additions I’ve made to my emacs configuration, so I’ll go through a number of them here. These are all passive minor modes; once you set them up, they work automatically. You don’t have to remember any keybindings to use them, or activate them in a buffer; they just work.

Mic-Paren

mic-paren is a package that enhances emacs’ parenthesis highlighting. It recognizes escaped parentheses, as well as matching parentheses both in front and back of point. It also has other options, such as highlighting other delimiters and the containing s-expressions. You’ll have to download this from http://www.emacswiki.org/emacs/download/mic-paren.el and then activate it with:

(require 'mic-paren)
(paren-activate)

Pager

Pager is a library for fixing emac’s paging. Without it, doing a page down followed by a page up has no guarantee of moving point back to it’s original location. Download pager from wget http://user.it.uu.se/~mic/pager.el and activate it with:

(require                  'pager)
  (global-set-key "\C-v"	'pager-page-down)
  (global-set-key [next]	'pager-page-down)
  (global-set-key "\ev"     'pager-page-up)
  (global-set-key [prior]	'pager-page-up)
  (global-set-key '[M-up]	'pager-row-up)
  (global-set-key '[M-kp-8] 'pager-row-up)
  (global-set-key '[M-down] 'pager-row-down)
  (global-set-key '[M-kp-2] 'pager-row-down)

which-func

Which-func is a minor-mode that will add the function point is inside to the mode-line. This is mainly useful if you are looking at large functions, but it could also be nice if you use vertical splits. Which-func mode is built into emacs, so you have to go through very little work to enable it.

(setq which-func-modes t)
(which-func-mode 1)

winpoint

Winpoint is a package that enhances multiple-window viewing of a file. Normally, if you have two windows open to the same buffer at different locations, and temporarily use one window to view something else, when you return to the original buffer it will have point be located where the other window’s point is. This is usually not the behavior I want, and winpoint fixes this so that point is kept on a per-window basis. To get this, you must first put winpoint.el, available from wget http://www.emacswiki.org/emacs/download/winpoint.el, in your .emacs.d directory, and then add the following to your .emacs file:

(require 'winpoint)
(winpoint-mode t)

dired-isearch

This package will replace the regular isearch commands with commands that will only match filenames when in dired-mode. This is normally what I want to do, so this package just makes it slightly easier. It is available from http://emacswiki.wikiwikiweb.de/cgi-bin/wiki/download/dired-isearch.el, with the following required in your .emacs file:

(require 'dired-isearch)
  (define-key dired-mode-map (kbd "C-s") 'dired-isearch-forward)
  (define-key dired-mode-map (kbd "C-r") 'dired-isearch-backward)
  (define-key dired-mode-map (kbd "ESC C-s") 'dired-isearch-forward-regexp)
  (define-key dired-mode-map (kbd "ESC C-r") 'dired-isearch-backward-regexp)

highlight-parentheses

The last package I’ll cover today is for highlighting parentheses around the current point, not just ones matching the current point. This is useful for telling you when the containing expression starts or ends, for example. This requires a bit more customizations than the other ones, for two reasons:

  • It does not define a global mode, so you have to do this yourself.
  • The default colors I find are pretty terrible.

Given that, my customizations are as follows:

(require 'highlight-parentheses)
(defun turn-on-highlight-parentheses-mode ()
(highlight-parentheses-mode t))
(define-global-minor-mode global-highlight-parentheses-mode
  highlight-parentheses-mode
  turn-on-highlight-parentheses-mode)
(global-highlight-parentheses-mode)
(setq hl-paren-background-colors '("green"))

The “green” is just what I am using now; change that to whatever color you wish.

Emacs, Overlays, and Point-Motion

Tuesday, February 9th, 2010

One feature that’s missing from Emacs is the ability to run a hook when the point moves into an overlay. Now, there are several use cases for this; the most obvious one being displaying messages when you are looking at some specific part of the code. For example, flymake displaying the specific syntax error when the point is in the highlighted part. This could also be used for completions.

Now, there are ways to do similar things in emacs. You can assign functions to the ‘point-entered and ‘point-left text properties, and they are run when the point enters the function. However, in some cases it would be easier to use an overlay; for example, js2-mode uses an overlay for the error checking, but must set this property on the actual text. This is clearly suboptimal; b oth setting up and cleaning up overlays requires remembering about this text property. Help-at-pt.el can also help, but it can only display messages, and only display arbitrary actions.

This isn’t a problem that can be solved on the emacs-lisp side of things; a behavior like this needs to be added at the C level of emacs. I decided to delve in and investigate how you’d accomplish this; I ended up finding the correct places to add this functionality - a function called command_loop_1. There was additional helper code in other places, but the code that actually called the functions was here.

However, after discussing it on the emacs development mailing list, the interface for defining a hook got uglier to make it more general. Currently, the hook takes five arguments; the last point position, the current point position, the overlay/text property that contains the hook, the previous buffer, and the previous window. This is to support being able to support executing when the point leaves because it went to a different window, etc. The hook is called on every point motion; entering or leaving the overlay/text property, including by way of buffer or window switching, and motion inside the property.

Combined with the ugliness, there was no real use case I could think of other than the display, where there are the above-mentioned workarounds. The patch is currently languishing on the bug tracker; if you want it, please comment on the bug report at http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=5397. My patch is also located at point-motion.patch, if you wish to apply it and try it on your emacs system. To use it once you’ve patched your emacs, just add functions to the ‘point-motion text or overlay property.

Emacs Bugfixes: Dirtrack Errors, Python Highlighting, and VC-Revert Window Clobbering

Monday, February 1st, 2010

I’ve recently gotten involved in the emacs development community. I’m not a commiter or anything, but I’m working on improving Java support for cc-mode, have a lrage patch(that is unfortunately currently in limbo), and have been submitting bugfixes for problems I’ve been running into. This post describes the fixes for a few of these bugs that you can add to your emacs configuration and fix right now.

The first patch is to dirtrack. If you have a somewhat incorrectly-specified regexp for your shell, dirtrack will sometimes eat output from your shell buffer and error out. Specifically, this happens when a non-existent directory is matched. This is clearly not what you want; instead, in most cases I want dirtrack to just ignore this, not switch the buffer, and proceed as normal. The following function does this, although it still messages you about an incorrect match:

(defun dirtrack (input)
  "Determine the current directory by scanning the process output for a prompt.
The prompt to look for is the first item in `dirtrack-list'.
 
You can toggle directory tracking by using the function `dirtrack-mode'.
 
If directory tracking does not seem to be working, you can use the
function `dirtrack-debug-mode' to turn on debugging output."
  (unless (or (null dirtrack-mode)
              (eq (point) (point-min)))     ; no output?
    (let (prompt-path
          (current-dir default-directory)
          (dirtrack-regexp    (nth 0 dirtrack-list))
          (match-num	      (nth 1 dirtrack-list))
          ;; Currently unimplemented, it seems.  --Stef
          (multi-line	      (nth 2 dirtrack-list)))
      (save-excursion
        ;; No match
        (if (not (string-match dirtrack-regexp input))
            (dirtrack-debug-message
             (format "Input `%s' failed to match `dirtrack-list'" input))
          (setq prompt-path (match-string match-num input))
          ;; Empty string
          (if (not (> (length prompt-path) 0))
              (dirtrack-debug-message "Match is empty string")
            ;; Transform prompts into canonical forms
            (setq prompt-path (funcall dirtrack-directory-function
                                       prompt-path)
                  current-dir (funcall dirtrack-canonicalize-function
                                       current-dir))
            (dirtrack-debug-message
             (format "Prompt is %s\nCurrent directory is %s"
                     prompt-path current-dir))
            ;; Compare them
            (if (or (string= current-dir prompt-path)
                    (string= current-dir (abbreviate-file-name prompt-path)))
                (dirtrack-debug-message (format "Not changing directory"))
              ;; It's possible that Emacs will think the directory
              ;; won't exist (eg, rlogin buffers)
              (if (file-accessible-directory-p prompt-path)
                  ;; Change directory
                  (and (shell-process-cd prompt-path)
                       (run-hooks 'dirtrack-directory-change-hook)
                       (dirtrack-debug-message
                        (format "Changing directory to %s" prompt-path)))
                (progn (message "Directory %s does not exist" prompt-path) input)))
            )))))
  input)

Another small patch is to python.el’s syntax highlighting. In the current form, def and class are only highlighted as keywords when the function/class name is started. This is clearly wrong, and led to me spending a few minutes looking up python method declaration syntax, as it looked like ‘def’ wasn’t a keyword. To fix this, put this in your .emacs file:

 
    (,(rx symbol-start "None" symbol-end)	; see � Keywords in 2.5 manual
     . font-lock-constant-face)
    ;; Definitions
    (,(rx symbol-start (group "class") (? (1+ space) (group (1+ (or word ?_)))))
     (1 font-lock-keyword-face) (2 font-lock-type-face))
    (,(rx symbol-start (group "def") (? (1+ space) (group (1+ (or word ?_)))))
     (1 font-lock-keyword-face) (2 font-lock-function-name-face))
    ;; Top-level assignments are worth highlighting.
    (,(rx line-start (group (1+ (or word ?_))) (0+ space) "=")
     (1 font-lock-variable-name-face))
    ;; decorators
    (,(rx line-start (* (any " \t")) (group "@" (1+ (or word ?_ ?.))))
     (1 font-lock-type-face))
    ;; Built-ins.  (The next three blocks are from
    ;; `__builtin__.__dict__.keys()' in Python 2.5.1.  Now modified
    ;; for Python 2/3 compatibility.)  These patterns are debateable,
    ;; but they at least help to spot possible shadowing of builtins.
    (,(rx symbol-start (or
                        ;; exceptions
                        "ArithmeticError" "AssertionError" "AttributeError"
                        "BaseException" "DeprecationWarning" "EOFError"
                        "EnvironmentError" "Exception" "FloatingPointError"
                        "FutureWarning" "GeneratorExit" "IOError" "ImportError"
                        "ImportWarning" "IndentationError" "IndexError" "KeyError"
                        "KeyboardInterrupt" "LookupError" "MemoryError" "NameError"
                        "NotImplemented" "NotImplementedError" "OSError"
                        "OverflowError" "PendingDeprecationWarning" "ReferenceError"
                        "RuntimeError" "RuntimeWarning" "StandardError"
                        "StopIteration" "SyntaxError" "SyntaxWarning" "SystemError"
                        "SystemExit" "TabError" "TypeError" "UnboundLocalError"
                        "UnicodeDecodeError" "UnicodeEncodeError" "UnicodeError"
                        "UnicodeTranslateError" "UnicodeWarning" "UserWarning"
                        "ValueError" "Warning" "ZeroDivisionError") symbol-end)
     . font-lock-type-face)
    (,(rx (or line-start (not (any ". \t"))) (* (any " \t")) symbol-start
          (group (or
                  ;; callable built-ins, fontified when not appearing as
                  ;; object attributes
                  "abs" "all" "any" "bool"
                  "chr" "classmethod" "cmp" "compile" "complex"
                  "copyright" "credits" "delattr" "dict" "dir" "divmod"
                  "enumerate" "eval" "exit" "filter" "float"
                  "frozenset" "getattr" "globals" "hasattr" "hash" "help"
                  "hex" "id" "input" "int" "isinstance" "issubclass"
                  "iter" "len" "license" "list" "locals" "map" "max"
                  "min" "object" "oct" "open" "ord" "pow" "property" "quit"
                  "range" "repr" "reversed"
                  "round" "set" "setattr" "slice" "sorted" "staticmethod"
                  "str" "sum" "super" "tuple" "type" "vars"
                  "xrange" "zip")) symbol-end)
     (1 font-lock-builtin-face))
    (,(rx symbol-start (or
                        ;; other built-ins
                        "True" "False" "Ellipsis"
                        "_" "__debug__" "__doc__" "__import__" "__name__") symbol-end)
     . font-lock-builtin-face)))

The last small patch is a fix for vc-revert-file. Right now, if you have a split window and call vc-revert-file, your other window will be closed in the process of reverting the file. This is clearly wrong; the following function replaces vc-revert-file to leave your window configuration alone after reverting the file.

(defun vc-revert () "This asks for confirmation if the buffer contents are not identical to the working revision (except for keyword expansion)." (interactive) (save-window-excursion (let* ((vc-fileset (vc-deduce-fileset)) (files (cadr vc-fileset))) ;; If any of the files is visited by the current buffer, make ;; sure buffer is saved. If the user says `no', abort since ;; we cannot show the changes and ask for confirmation to ;; discard them. (when (or (not files) (memq (buffer-file-name) files)) (vc-buffer-sync nil)) (dolist (file files) (let ((buf (get-file-buffer file))) (when (and buf (buffer-modified-p buf)) (error "Please kill or save all modified buffers before reverting"))) (when (vc-up-to-date-p file) (unless (yes-or-no-p (format "%s seems up-to-date. Revert anyway? " file)) (error "Revert canceled")))) (when (vc-diff-internal vc-allow-async-revert vc-fileset nil nil) (unless (yes-or-no-p (format "Discard changes in %s? " (let ((str (vc-delistify files)) (nfiles (length files))) (if (< (length str) 50) str (format "%d file%s" nfiles (if (= nfiles 1) "" "s")))))) (error "Revert canceled")) (delete-windows-on "*vc-diff*") (kill-buffer "*vc-diff*")) (dolist (file files) (message "Reverting %s..." (vc-delistify files)) (vc-revert-file file) (message "Reverting %s...done" (vc-delistify files))))))