# Speeding up org-publish

I publish my [[Digital garden]] using [[org-publish]]. ([[Publishing org-roam via GitLab CI]]).

It is shockingly slow. Around when I first introduced the pipeline, it took about 6 minutes - pretty slow.  Currently, as of Feb 2022 with over 3000 org files, it's consistently taking around **30 minutes**!!!  This is horrendous.  


## Speed over time

I can pull out the builds time from my gitlab pipeline runs.
I can pull out the number of files over time from git (https://blog.benoitblanchon.fr/git-file-count-vs-time/)

![[Speed_over_time/2022-02-05_12-55-25_screenshot.png]]

It judders around a bit, likely down to the fact that gitlab runners can just take different amount of time, but it's a **reasonably** linear trend.  So it kind of looks like the more files, the more build time.  So the amount of time publishing a file in the problem.


## Profiling

Bit of a tip here on profiling the publish operation:  [Speeding Up Org Mode Publishing](https://bastibe.de/2014-05-07-speeding-up-org-publishing.html) 

I did that, and a big chunk of the process (22%?) seems to be coming from web-mode hooks.


## Some things to try


### Turning off hooks

I've tried a couple of ways of turning off off web-mode hooks.

```emacs-lisp
(rassq-delete-all 'web-mode auto-mode-alist)
(fset 'web-mode (symbol-function 'fundamental-mode))
```

(see https://emacs.stackexchange.com/questions/21381/how-to-never-have-a-major-mode-enabled-automatically)

```emacs-lisp
(defun org-publish-ignore-mode-hooks (orig-func &rest args)
  (let ((lexical-binding nil))
    (cl-letf (((symbol-function #'run-mode-hooks) #'ignore))
      (apply orig-func args))))
```

```emacs-lisp
(advice-add 'org-publish :around #'org-publish-ignore-mode-hooks)
```

It might be doing **something**, but I haven't a proper before and after profile of that yet.


### Suppressing output messages

Every single file that is published outputs a message like:

> Publishing X with org-html-publish-to-html

It's quite nice to see progress through the files, but not really neccesary and output is usually a performance problem.

```emacs-lisp
(advice-add 'org-publish-all :around 'silence)

(defun silence (orig-func &rest args)
  (let ((inhibit-message t))
    (apply orig-func args)))
```

This reduced the time from about 30 to 24 minutes.

It would be useful to see error messages though - I'll see if those still get output when I encounter one.


### Incremental builds

One problem is potentially that the rebuild is from scratch each time.  If I could make it incremental, I could to some extent bypass the problem of org-publish slowness, as I'd only be building a few files each time.  But goes a bit against the principle of CI builds.

I make be able to make use of pipeline caching for this.
What would I cache?


## Resources

-   [How to speed up org export/publish? : emacs](https://www.reddit.com/r/emacs/comments/mmdeei/how_to_speed_up_org_exportpublish/)