One of the sites I maintain has a couple of VOBs (Versioned Object Base, it’s called repository in other version control systems) which got corrupted. Someone from NAS team (it’s not your home NAS system) tweaked with NFS parameters, and locked out the VOB server for a couple of seconds. It’s not nice.

Anyways, the server reboots every now and then, to add more spice.

To serve users better, I got the problem report two days later (a problem report in normal severity should be fixed in 5 business days). You know, you have to fight with Level 1, then comes Level 2 where they think they can solve the issue by putting them aside.

Fortunately, I keep a week-long nightly snapshots around, and I was able to restore damaged vobs. I had no issues with the first one, but the second vob had local changes since the last good backup.

Let me digress a bit to tell you how ClearCase keeps replicated data up to date between nodes.

First off, the core principle is quite simple: you can write your changes, but you can only read others’ changes. It goes down to ClearCase’s core change principle: you have to check out files you want to modify, and it makes read-only for others.

How you can support multiple users then? The answer is branching. Everybody creates his/her own topic branch, which then will get merged back to baseline. To broaden the solution back to multisite usage, every version, branch, label is mastered by a replica, where it can be written or modified. This simple principle is the core of base ClearCase, which then causes trouble in various places.

OK, we have a couple of replicas, now update them. For this, the vob maintains a log of operations (a.k.a. oplog) for every replica. The local replica keeps its oplog tidy, but it also stores all the other replicas’ oplogs, in case we have to update a replica with another’s changes. Yes, you can create your own update topology.

To make things more quirky, the local replica keeps track of oplog counter (also called epoch number) of what it thinks other replicas have. This is how a replica can update a replica with another one’s changes.

So far it’s easy to follow, but here comes the problem: when a replica updates another, it assumes the update will be imported, and it increases the appropriate counters, no matter the packets reached destination. This fragile system causes the most common problem with ClearCase: synchronization issues.

You can fight them with updating your copy of a remote replica’s epoch list, by manually updating it, or by using restorepackets. In the latest version, you can even have a diagnostic tool which shows you what command should be run on the other replica. These methods don’t solve the problem once and all, but they give job security for a ClearCase admin.

 Now I can be a bit more technical: the second vob’s epoch number was higher than the restored version’s. This means we have to switch to restorereplica mode, where we ask others to give our changes back. Fortunately ClearCase has an admin guide which describes what to do in a situation like this. Integrity check of the database (where relations are stored), vob source pool check (where file differences are stored), fix all issues, and then switch to restorereplica, and wait for the updates from all other replicas.

Nice. Source pool checking pointed to 6 containers being missing. Sure enough all of them were in place. Nevermind, I fixed them, and then checkvob returned with no errors. Let’s jump right into restorereplica!

I started with France. Packets back and forth, looks right… or not? Importing says there is still a missing container.

At this point the server crashed, and got rebooted.

I started all the tests over. Database check went well, but checkvob showed 6 missing containers again, but now with really missing files. Oh my. Switch back from normal mode, fix the holes, turn on restorereplica again, then send out updates to replicas. Done.

I started with France again. Packets back and forth. Now there’s another issue in importing: the source container I just restored has some versions which are not in the database.

How can I remove a version from a source container?

It turned out the format is not too difficult. A line starts with caret (^) is a command. We have a couple of commands:

• ^E: element, it tells you the element’s UUID
• ^V: version. After version’s UUID, it contains the branch number (in hex), and version number (hex).
• ^B: branch. Branch UUID, branch number, and some other data.
• ^I: insert lines in a version. Branch number, version number, and number of lines. Then, the actual lines.
• ^D: delete lines from a version. Branch number, version number, and the number of lines to be deleted.

It has some other commands as well, but they’re not important if you want to hack a container.

I solved all my restoring needs, but this hack was the tip of the iceberg.

And it happened, soon enough. All packet import says there’s an oplog divergence: the replica thinks the change was in a different oplog entry. Now every replica is stuck, no one can update any other.

Now you can either call IBM, or you put an axe on the table.

After a long meeting we decided option two will be the solution. We had more than 1000 changes in a replica, which should be sent to other sites, and we don’t have weeks to figure out which bit went to the wrong place.

When you have a divergence, the easiest way is to get rid of the whole oplog. Just choose a replica which will survive, remove other replicas one by one, and when the last remote replica gets removed, your vob will return to it’s original state. No oplogs, no export_sync records (it is a way to reset a replica’s epoch matrix, but may I ask why we have two different methods for this?), no mastership.

To do this, all you have to do is to make other replicas obsolete, and then remove them at the selected site.

Yes, sure.

At this point I learned a vob database’s journal can hold at most 2GB transaction data, and sometimes it’s not enough to remove the last replica. Why? Because of the enormous amount of oplogs and export_sync records.

Sadly it never told me. It just hung. After a couple of hours I got suspicious. I’ve listened into all the processes in question with truss, but all of them were just sleeping. Aha, deadlock. Also cleaning up corrupted transaction file droppings, which was all described in a technote.

At this point I felt 2.7GB for a database is a bit too much. I dumped then loaded the database, and it reduced to under 1GB. Well, ten years is a long time to fragment a database.

Here comes another fancy feature of ClearCase: from time to time a lot of unnecessary data gets collected. Nobody really cares about who and when locked the vob (we have to lock the database every day during backup), or who applied labels. To remove these data we have a vob_scrubber utility, which weeds our garden.

Oplogs are kept forever. Export sync records kept forever. I changed the rules to keep only 7 days, but scrubber failed miserably.

In a technote I never seen before I found the solution: peel off oplogs and export sync records. Start with 120 days, then go on to 90, 60, and 30 days. Then finish off with a 7 days amount of export sync records being kept, and then, you’ll be able to remove the latest replica.

Now I’m at this point, but another 9 hours passed, and I need a cigar.

$ echo rvm 1.9.2 > .rvmrc
$ rvm rvmrc load

We’ll going to store project-related binaries in bin folder, why don’t we add this to the search path? This line is a good candidate of a .bashrc or .zshenv line:

$ export PATH="./bin:$PATH"

UPDATE: I have to admit this is the weakest part of the solution, or, in other words, this is the least elegant way of doing things. However, we use bundler anyways, why don’t we use it? Thus, open our development console with this command instead:

$ bundle exec $SHELL

This command just lets you peek inside the bundle, with gems, PATH settings, and such funny things. Just to the point, and you don’t have to worry again of malicious binaries in world-writable directories (funny things in /tmp/bin/ls for example). I have a couple of projects in parallel, why should we store all these gems multiple times?

$ mkdir ~/.bundle
$ ln -s ~/.bundle vendor/ruby

Now, we can bundle our stuff:

$ bundle --path vendor --binstubs
$ bundle --path vendor

(UPDATE: --binstubs is not required since we use bundle exec $SHELL). You have to keep bundled environments out of versioned files:

$ (echo bin/; echo vendor/ruby/) > .gitignore
$ echo vendor/ruby/ > .gitignore

That’s it (UPDATE: putting bin/ to .gitignore is unnecessary now). I found only one issue with this setup: I’m using guard for a while, and while nearly everything works well, I was not able to get guard-bundler working. Until a recent update. The only issue was this: guard (just like all the commands living in ./bin) runs in bundler’s environment, however, in order to run bundle command, you actually have to have it installed. However, bundler is not installed inside the bundled environment. As it turns out the answer is easy, but not intuitive: let’s install bundler into the bundled environment:

$ bundle exec gem install bundler
$ gem install bundler

(UPDATE: bundle exec is not needed in development console.)

Now we have the executable inside, which guard-bundler can run in a Bundler::with_clear_env block (which resets environment as it was outside), and it can install gems, flowers, unicorns, double rainbows. With these settings we get pretty much the same experience as a non-rvm environment, or a dedicated gemset for every project. Sometimes, when .bundle becomes too big, I just dump it and start over. It doesn’t take too long anyways. To wrap up, we achieved the following:

1. We use rvm
2. However, we got rid of it’s bundler helpers
3. We don’t rely on the bundler pool using executables / scripts
4. … but we have a pool, and we don’t have multiple copies the same gems (ssd is still not cheap)
5. Most shortcomings of vendoring gems are hidden

Enjoy.

Látván a remek Harvest, FreshBooks, vagy az európai InvoiceMachine rendszereket bizony elszomorító, hogy Magyarországon miért kellett körbebástyázni ezt a területet egy mikrovállalkozásnak nem betartható szabályokkal, amikhez hasonló más országokban (értsd: Egyesült Királyság és Amerikai Egyesült Államok — elvégre sznob vagyok) nem léteznek.

Azonban ha jobban végiggondoljuk az egészet, a bástyasor nem más, mint egy foghíjas szita. Tehát szerintem az alábbi mód teljesen helyes:

1. A vállalkozó elindítja az Office 2010-et, kiválaszt egy számla template-et.
2. Előző kibocsátott számláira hagyatkozva új számlasorszámot választ, és készít egy számlát a megrendelőnek. Ebből két sorszámozott példányt készít: egy 1. példányt PDF-be nyomtat, egy 2. példányt pedig papírra. Mindeközben készít egy 2. példányt is PDF formátumban.
3. Az 1. példányt ízlése szerint GPG kulcsával aláír, és esetleg a megrendelő kulcsával titkosít. Ezt a példányt elküldi email-ben a megrendelőnek.
4. A 2. példány papír változatát eljuttatja a könyvelőjének, a PDF változatot pedig saját dokumentumtárában elmenti (lehet, hogy erre 3. példányt kéne használni?)
5. A megrendelő megkapja az első példányt, és a szerződésben foglaltak szerint kinyomtatja, és beteszi a saját könyvelésébe.

Mi akadályozna meg ebben? A PM rendelet? Lássuk:

Számítástechnikai eszköz útján, papírra nyomtatott számla adóigazgatási azonosításra való alkalmasságának szabályai

1/E. § (1) Számítástechnikai eszköz útján előállított és papírra nyomtatott számla abban az esetben alkalmas adóigazgatási azonosításra, ha szigorú számadás alá vonása úgy valósul meg, hogy

a) a számla kibocsátására szolgáló számítástechnikai program (a továbbiakban: számlázó program) – a (4)-(6) bekezdésben meghatározott eltérésekkel – kihagyás és ismétlés nélkül, folyamatosan biztosítja a sorszámozást, továbbá

A számítástechnikai program nem más, mint az Office 2010, ami teljesen alkalmas kihagyás és ismétlés nélkül biztosítani a sorszámozást, sőt, még annál sokkal többet is tud. Persze nem automatikusan, de ez nincs is megkövetelve. Az alkalmasság pedig ugye megvan.

b) a számlapéldányok hiánytalan elszámolása az 1/F. § és az 1/H. § szerint biztosított.

Ezeket majd talán ott.

(2) A számla kibocsátójának a számlázó program olyan dokumentációjával kell rendelkeznie, amely tartalmazza a program működésére, használatára vonatkozó részletes leírást, valamint a program készítője vagy jogutódja által a számla kibocsátójának címzett írásos nyilatkozatát arról, hogy az maradéktalanul megfelel a vonatkozó jogszabályi előírásoknak.

Az Office 2010 dokumentációja, ami mellesleg csak elektronikusan elérhető, a szoftver minden részletére kiterjed. Az írásos nyilatkozat pedig a szerződés, amiben az összes olyan jogszabályt feltüntetnek, amiben érintett az Office 2010. A PM rendeletet nem, mert ugye nem egy specifikus probléma megoldására készült a program, azonban minden olyan szabályozásnak megfelel az Office 2010, amelyben a szövegszerkesztőkre vonatkozó előírásokat tartalmazzák (nincs ilyen).

(3) A számlázó program a hiányos, hibás, megsemmisült vagy elveszett számlákat is rögzíti.

Rögzíti hát, Save As. Az mindent rögzít.

(4) A számlázó program az adóalany nem magyarországi adószámán [Áfa tv. 258. § (3) bekezdés b) és c) pont] történő számlakibocsátásához a belfölditől elkülönített sorszámtartományt határoz meg.

Ebben kifejezetten jó az Office. Azt írok oda, amit akarok, tehát mindenképpen más számlatartományban fogok sorszámot választani a külföldi számlák esetén.

(5) A sorszámozás folyamatossága nem sérül, ha

a) a csoportos adóalanyiságban [Áfa tv. 8. §] részt vevő tagok belső, egymás közötti kapcsolataiban a számlának nem minősülő egyéb számviteli bizonylat kibocsátása, illetőleg

b) a csoportos adóalanyiságban részt vevő tagok külső, harmadik személlyel szembeni kapcsolataiban a számla kibocsátása azonos sorszámtartományon belül történik.

Ezen szabályozások betartása sem nehéz.

(6) A sorszámozás folyamatossága nem sérül akkor sem, ha a számla és az Áfa tv. 165. § (1) bekezdés a) pontjában meghatározott számviteli bizonylat kibocsátása azonos sorszámtartományon belül történik.

Szintén, sőt, köszönjük a könnyebbséget.

1/F. § (1) (nem hatályos)

(2) A számítástechnikai eszköz útján előállított és papírra nyomtatott számla kibocsátónál maradó példánya – papírra nyomtatás helyett – elektronikus adatállományként is megőrizhető, feltéve, hogy a megőrzés a digitális archiválás szabályairól szóló 114/2007. (XII. 29.) GKM rendelet rendelkezései szerint, annak 3. § a) vagy b) pontjában meghatározott mód valamelyikeként történik.

A lényeg a „helyett” szóban van. Nem helyettesítem a könyvelésben található számlát, hiszen azt papír alapon eljuttatom a könyvelőnek (bár neki is van nyomtatója), és ugyanígy fog járni a megrendelőhöz érkező példány is. Nálam pedig pusztán referenciaként szolgál, nem számlaként.

(3) A (2) bekezdés szerint megőrzött adatállomány a számítástechnikai eszköz útján előállított és papírra nyomtatott számla elektronikus változata, amely nem minősül elektronikus úton kibocsátott számlának [Áfa tv. 175. §].

Hoppá, hiába tartottam volna csak elektronikusan, az sem számla? Sosem baj, hiszen papírként van meg a könyvelőnél.

Elektronikus úton kibocsátott, illetőleg az ilyen alapon rendelkezésre álló számla adóigazgatási azonosításra való alkalmasságának szabályai

1/G. § Az elektronikus úton kibocsátott, illetőleg az ilyen alapon rendelkezésre álló számla adóigazgatási azonosítására az 1/E. §-t, valamint az 1/H. §-t kell megfelelően alkalmazni.

Rekurzió: lásd rekurzió. Amúgy sem vonatkozik ránk, hiszen az elektronikus úton kibocsátott valamilyen oknál fogva a jogalkotóknál teljesen más értelmezésben él, mint az összes többi emberben. Ez áll az ÁFA törvény értelmező rendelkezéseiben:

5. elektronikus úton kibocsátott számla: a termék beszerzője, szolgáltatás igénybevevője részére az adatok vagy – digitális tömörítés felhasználásával – adatállományok elektronikus úton történő továbbítása, személyes rendelkezésre bocsátása telefonvezetékes, rádiós, optikai vagy egyéb elektromágneses rendszeren keresztül, elektronikus adatfeldolgozás céljára;

TEHÁT ha nem elektronikus adatfeldolgozás céljára továbbítom elektronikusan vagy személyesen a PDF-et, nem minősül digitális úton kibocsátott számlának! A nyomtatás pedig itt nem elektronikus adatfeldolgozás, mert csak a belőle készült papír alapú számlát fogják feldolgozni.

A számítástechnikai eszköz útján, papírra nyomtatott számla és az elektronikus úton kibocsátott, illetőleg az ilyen alapon rendelkezésre álló számla adóigazgatási azonosításra való alkalmasságának közös szabályai

1/H. § (1) Az elektronikus úton kibocsátott, illetőleg az ilyen alapon rendelkezésre álló számla és az 1/F. § (2) bekezdés szerint megőrzött számla helyszíni ellenőrzése során az adóhatóság részére

a) biztosítani kell a számla olvashatóságához szükséges technikai eszközöket,

b) rendelkezésre kell bocsátani az a) pontban említett eszközök használatára vonatkozó dokumentációt, továbbá

c) meg kell adni az a) pontban említett eszközök használatához szükséges felvilágosítást.

Hmm. Adjak nekik szemüveget? Mi ez a slendriánság az 1/E. b)-ben?

(2) Az adóhatóság jogosult a számla kiállításának folyamatát úgy is ellenőrizni, hogy felügyelete alatt a számla kiállítója – próbajelleggel – állít ki számlát.

Bárkit szívesen megtanítok a szövegformázás alapjaira Office 2010 alatt, amennyiben hajlandó a tanfolyamot kifizetni.

Na, ha nem a PM rendelet, akkor bizonyára az ÁFA törvény. Ott az érdekes rész a 175.§-nál kezdődik:

175. § (1) Számlát elektronikus úton kibocsátani, illetőleg ilyen alapon rendelkezésre álló számlára e törvényben szabályozott jogot alapítani, kizárólag abban az esetben lehet, ha a számla és az abban foglalt adattartalom sértetlensége és eredetiségének hitelessége biztosított.

A számlát papír alapon bocsátjuk ki, melyben felhatalmazást adok az ügyfélnek, hogy a kibocsátás megvalósítását (ti. a számla kinyomtatását) az általam rendelkezésére bocsátott adattartalommal (ti. a PDF alapján) tegye meg. Amennyiben *bármelyikünknek* kételyei támadnának az adattartalom helyességéről, azt mi, egymás között a remek GnuPG titkosító rendszerrel tudjuk mégis biztosítani. Ismétlem, maguk a GPG-titkosított és aláírt fájlok nem minősülnek elektronikusan kibocsátott számlának, csak elektronikusan kibocsátott számla adattartalomnak, amelyből később egy konvencionális számla fog születni.

(2) Az (1) bekezdésben említett követelménynek való megfelelés érdekében az elektronikus úton kibocsátott számlát

a) az elektronikus aláírásról szóló törvény rendelkezései szerinti, legalább fokozott biztonságú elektronikus aláírással és minősített szolgáltató által kibocsátott időbélyegzővel kell ellátni; vagy

Csakcsupán kérdés, nem tartozik ide: minek a minősített szolgáltató által kibocsátott időbélyegző, ha már fokozott biztonságú elektronikus aláírással is elláttuk? A papír alapú számla nem hazudik, de az elektronikus meg mint a vízfolyás? A számla egyik alapvető eleme a kibocsátás dátuma, ami a dokumentum összes többi elemével együtt digitálisan alá van írva. Tehát akár egy pici részét megváltoztatják, már az aláírás nem fog stimmelni. Ja persze, megvan: nem bízunk senkiben. Üdvözlet a való világban: a számla egy bizalom megtestesülése. Nem mellesleg attól, hogy elektronikusan lett kiállítva, még mindig szükséges, hogy mindkét félnek  rendelkezésére álljon egy-egy példány.

Az egy másik dolog, hogy mit jelent a fokozott biztonságú elektronikus aláírás. GPG-t a saját számítógépemen (csak hogy örüljenek, addig kikapcsolom a wifit, és csak utána dugom be a külső diszket, amin a titkos kulcsom van. Persze lehet ragozni, de minek). Nincs minősítési követelmény.

b) az elektronikus adatcsererendszerben (a továbbiakban: EDI) elektronikus adatként kell létrehozni és továbbítani.

(3) A (2) bekezdés b) pontja alkalmazásának feltétele, hogy a számlakibocsátásra kötelezett és a termék beszerzője, szolgáltatás igénybevevője előzetesen és írásban megállapodjon az EDI alkalmazásáról és használatáról.

(4) A (2) bekezdés b) pontja alkalmazása esetében a számlakibocsátásra kötelezett köteles gondoskodni arról is, hogy havi rendszerességgel az adott hónapban kibocsátott számlákról papíron kiegészítő összesítő jelentés készüljön, és azt a termék beszerzője, szolgáltatás igénybevevője megkapja.

(5) Külön jogszabály az (1) bekezdésben említett követelménynek való megfelelés érdekében az elektronikus úton kibocsátott számlára egyéb rendelkezéseket is megállapíthat.

Ó hogyne. Majd ha áttérünk elektronikusan kibocsátott és tárolt számlákra, amit majd az állam rendelkezésünkre ad. Addig maradunk a papírnál, hiszen még mindig a környezet pusztítását követeli meg a törvény, és alternatívaként egy betarthatatlan jogszabály-folyóhomokot biztosít. Ó nem, ott is kell papír alapú kiegészítő összesítő jelentés. De ciki.

Ezek alapján egy Word-ből vagy Excel-ből nyomtatott számlát teljesen hitelesnek lehet elfogadni. Sajnos ettől még mindig nem tudok InvoiceMachine-en keresztül számlázni, de már ez is valami.

That’s true. We started web hosting in small scale. Now we have much more performant machines, even if we’re still in the same scale. And then, we realized we don’t need this amount of CPU, that amount of RAM, but we still need our stuff separated. We started using VPS.

My usual wise but entirely unhelpful answer was like use nginx. Being an apache user, I couldn’t even tell whether it will fit in.

Hell, it’s about time.

I used to be a hard core FreeBSD user, but currently I maintain only two servers. Both run Ubuntu. I took over the first one, and I moved the second one from a failing system to a VPS. In a hurry. No choice.

Then, I felt it’s time to update my wiki server in the company I work for, which ran FreeBSD 7.0. I chose Ubuntu Server 10.10 for that.

I have thousands of answers why I switched my beloved multi-jail configuration to a single instance (what, no need to ‘ssh tcb’ to see web server logs anymore?), but among them there are interesting stuff like I was unable to compile python 2.7, or Phusion Passenger (a.k.a. mod_rails).

The long story short I switched (back) to Linux, and in the mean time I found an interesting application service for PHP, which replaces the last bastion of Apache requirement: mod_php.

Why it’s interesting, especially for a rails guy? For a ton of reasons: I have users who run a lot of services on the boxes I mentioned earlier, but most of them are WordPress, a couple of MediaWiki instances, and some other stuff, all of them are written in PHP.

We can’t deny PHP support.

Enginex screaming

(sorry for the second bad starcraft2 reference)

In Ubuntu, we can use nginx from the system, but we can use the one which comes with Passenger. Using the latter is better, because U10.10 contains nginx/0.7.67 (legacy version), but Passenger downloads nginx/0.8.54 (stable version). I strongly suggest having both, using Ubuntu’s configuration framework (which is excellent), while using the newer nginx version.

First, install essential devel packages:

# apt-get install build-essential libcurl4-openssl-dev libssl-dev zlib1g-dev libpcre3 libpcre3-dev

And set up a ruby you’d like to use for rails deployments. For now system ruby is more than enough.

# apt-get install nginx ruby ruby1.8-dev rubygems1.8
# gem install --no-ri --no-rdoc passenger

For some reason, this installs all ruby-generated executables into /var/lib/gems/1.8/bin, but so be it, we’ll run it only once. Now let’s build our nginx:

# /var/lib/gems/1.8/gems/passenger-3.0.4/bin/passenger-install-nginx-module

Now you can select 1) to download, and you can choose /opt/nginx, we’ll use the executable only. A couple of ENTER pressed, and you’re done. At the very last screen you can see which lines should you include into http config. Save these two lines and add them to /etc/nginx/conf.d/passenger.conf if you want to use Passenger.

Then, modify /etc/default/nginx to start this new server instead of the default one:

DAEMON=/opt/nginx/sbin/nginx
DAEMON_OPTS="-c /etc/nginx/nginx.conf"

Configure nginx

The system default configuration is well written: there’s a default config in /etc/nginx/nginx.conf, you can tweak http settings by putting *.conf files into /etc/nginx/conf.d, and you can set up virtual servers in /etc/nginx/sites-available directory. Then, you can symlink these to sites-enabled like this:

# cd /etc/nginx/sites-enabled
# ln -s ../sites-available/YOURSITE
# /etc/init.d/nginx restart

Cool, ain’t it? Let’s set up a config for a mediawiki site. I have two (or more) subdirectories under the user-editable directory: ‘html’ for document root, and ‘apps’ for deployed applications. I also put ‘tmp’ or ‘upload’ for per-site tmp/upload directory, my users don’t share my tmp.

# cd /srv
# mkdir -p www/mediawiki/apps
# cd www/mediawiki/apps
<download it locally and unzip here to a versioned subdirectory>
# mkdir ../html
# cd ../html
# chown -R www-app: ..
# ln -s ../apps/mediawiki-<version> wiki
# cd wiki
<configure>

We have the files. Now let’s set up the web service:

in /etc/nginx/sites-available/<server name>:

server {
  server_name mediawiki.tld;
  root /srv/www/mediawiki/html;
  location / {
    index index.php index.html index.html;
  }
  location ~ \.php {
    fastcgi_split_path_info ^(.+\.php?)(/.*)$;
    fastcgi_pass unix:/srv/www/sockets/www-data.sock;
    includes fastcgi_params;
  }
  location ~ /\.ht {
    deny all;
  }
  location /wiki/ {
    try_files $uri /wiki/index.php;
  }
}

It’s quite a mouthful, let’s break them down:

• no listen 80: that’s the default, why should we bother?
• index index.php? Yes, we trigger fastcgi backend by .php ending, and not by a location. Therefore directory => index page should be handled by the http server instead of fastcgi.
• location ~ \.php: Please note we don’t tell .php should be at the URI’s end. We have to deal with PATH_INFO after all.
• no fastcgi_index index.php? It wouldn’t fire. Fastcgi settings are triggered by a URI which contains .php in it. Fastcgi_index runs only if $uri is a directory. Therefore it would run only if you have a directory with .php in it’s name, which is generally not a good idea.
• fastcgi_split_path_info: this line does the trick setting PATH_INFO, if specified.
• includes fastcgi_params: there is a fastcgi_params file in /etc/nginx, however it’s not complete. You can find my extra lines below.
• location /wiki/ -> try_files: every documentation tells it should be try_files $uri $uri/ … . However, as we already figured out, .php execution is triggered by explicite inclusion of ‘.php’ in the URI. Telling try_files ‘$uri/’ just confuses nginx, and index file searching would be skipped.
• try_files … /wiki/index.php: if we want to skip Mediawiki’s front controller (eg. index.php) from the URL, we can safely say, everything under /wiki can be handled as a PATH_INFO. This just does that. Location strips /wiki from the URI (wrt. location path setting), and this value is passed to /wiki/index.php as PATH_INFO. And in the end, .php location will handle the request. Please note this PATH_INFO setting is totally different from the one we set in .php location. This happens only if no .php has been specified, and the other one applies only to URIs which have .php in them. You can use either or both.
• no if (-f…)? If is evil. try_files does a much more decent job anyways.

Of course, it’s not a production ready config, you should have a conf.d/gzip.conf, Expires header to static assets, temporary directory for uploads, error handling, and the like. However this is something we can start working with.

We need three more settings in fastcgi_params file:

fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
fastcgi_param  PATH_INFO          $fastcgi_path_info;
fastcgi_param  PATH_TRANSLATED    $document_root$fastcgi_path_info;

The first line is required if the backend doesn’t run in chroot. The other two are required to set PATH_INFO-related settings to fastcgi backend.

OK, the hard part is done. Let’s configure php5-fpm.

PHP5 FastCGI Process Manager

In Ubuntu, an ‘apt-get install php5-fpm’ will install it. Quite simple.

It comes with a configuration framework too, which is very pleasing. You can find everything in /etc/php5/fpm. There are two files in this directory: main.conf (for fpm config), and php.ini (for php config). You can see two directories too, a conf.d (for extra php config, pointing to /etc/php5/conf.d), and pool.d (for extra fpm config).

I prefer configuring one pool per PHP user, which can be tightened more in conf.d server by server.

For example, this is a standard user’s pool file:

/etc/php5/fpm/pool.d/USERNAME.conf:

[username]
listen = /srv/www/sockets/USERNAME.sock
listen.owner = www-data
listen.group = www-data
listen.mode  = 0660
user = USERNAME
group = GROUP

catch_workers_output = true

pm = dynamic
pm.start_servers = 2
pm.max_children = 50
pm.min_spare_servers = 2
pm.max_spare_servers = 5
pm.max_requests = 500

It listens on a unix domain socket, which is proven to be way efficient than a TCP connection (even in localhost), and it sets listening socket ownership to web server’s account. We don’t want anyone else to mess with the data.

Then, it starts a very conservative server. It starts with two servers (+1 for controlling), and at most 50 is spawned. It kills unused servers, keeping 5 at most. It reaps servers which served 500 requests.

My mediawiki config for web service looks like this:

/etc/php5/conf.d/host-mediawiki.conf:

[PATH=/srv/www/mediawiki]
error_log = /var/log/fpm/mediawiki.log
log_errors = on
display_errors = off
open_basedir = /srv/www/mediawiki/html:/srv/www/mediawiki/tmp
upload_tmp_dir = /srv/www/mediawiki/tmp
session.save_path = /srv/www/mediawiki/tmp

Please note you have to add a similar config to every single symlink / alias to conf.d. Not doing so would open up the servers to rogue PHP runners Defend yourself then (put it to /etc/php5/conf.d/host-nonexisting.conf):

[PATH=/]
log_errors = off
display_errors = off
open_basedir = /non_existing_directory

We’re done. Now we have replaced apache2 with nginx, and we started serving PHP through php5-fpm, which allows us to run PHP as the serving user, and not as www-data. Oh, and there’s sugar on top: passenger lets us running ruby (rack) and python (wsgi) apps out of the box!

I ended up returning to Vlad. The journey to Inploy had it’s advantage though.

What a deployment system should do for us?

Problem: multiple destinations is a must. You might want to deploy to a production server. To a staging server. To a CI server. You name it. Solution: Vlad honors ‘to’ variable. Write your default config to config/deploy.rb, and then, specific config to config/deploy_#{ENV['to']}.rb.

Problem: maybe you have multiple environments (like development server runs passenger, but you run thin with god in staging and production environment). Solution: don’t set :app or :web options in vlad.rake (set them to nil), and load them in the appropriate deploy_*.rb files.

Cleanup or more setup is required at deployment side. Solution: local rake / thor tasks or generators to be run after setup / update.

Make every deployment independent. Solution: Vlad does this. It clones a vanity repo, and it copies data to a unique, timestamped directory.

Support bundler. Solution: Bundler supports Vlad instead.

Support RVM. It’s not a simple answer.

First, bash from FreeBSD ports doesn’t run any rc scripts when running remote commands via ssh. My answer was to switch to zsh. FBSD + ssh + zsh == RVM script is loaded. Another option is to wrap every remote task around with bash -lc “…”.

Second, Vlad’s tasks don’t honor .rvmrc trust prompts, and ‘rvm rvmrc trust <directory>’ is not not quite there. You have to call it twice for every deployment: one for ‘releases/DATE‘, and another for ‘current’. Even so, your trust file will grow by every deployment, but won’t shrink by removing any obsoleted deployments.

There are two possible solutions for this: you can either move project .rvmrc out of your repository (but we all know already, your repo should contain .rvmrc), or you can disable this annoying rvmrc trust altogether by adding ‘rvm_trust_rvmrcs_flag=1′ into your (or the deploy user’s) ~/.rvmrc file.

The article I mentioned above the author comes to the conclusion that project rvmrc files should not contain gemset information, only ruby version information. I have played with the idea, and I found it fits well into a deployment concept.

Gemsets and bundles are basically competing ideas

Of course there are differences, but basically they’re here to solve the very same problem: separating projects’ gem environment from others. RVM goes even further, and it changes bundler behavior to work into a gemset. However, bundler does it’s job better (wrt. dependency resolution), therefore it’s no need to have two gem environment separation technologies around.

Are gemsets have a use then? Of course they have. Using bundler also means you don’t have to pollute your default ruby environment with a ton of not needed gems. Sometimes, however, you want to use a tool or another, which are good to be around. Just like when you create a new rails project.

You don’t have to have the whole rails stack around in your (default) gemset: put it to rails3. And then, when you want to fire up a new project, just type in: ‘rvm 1.9.2@rails3 rails new …’

Set up your environments

That’s alright, but we still have two environments to sort out: a development and a deployment one. Why it matters? Because development environment is about being easily accessible, but your deployment env. is about to be robust. I strongly suggest vendoring your bundles in development:

bundle install --path vendor --binstubs --without production

In deployment, however, I’d use this:

bundle install --path ../../shared --deployment --binstubs --without test development

(you can tweak it for CI for example)

It means all of the binaries will be available in ./bin directory (eg. rake, thor, rspec, all of them). You might want to exclude bundle directory from Vim or Textmate:

Vim:

put ‘set wildignore+=vendor/ruby/**’ into .vimrc

TextMate:

Go TextMate -> Preferences, select Advanced, then Folder Preferences tab. Edit Folder Pattern:

• enclose patterns between starting ! and trailing $ into parens like this: !(.*/(…))$
• add ‘|vendor/ruby’ just before closing paren + dollar like this: !(.*(…)|vendor/ruby)$

You might want to repeat this for already created projects: open directory, select root directory in project drawer, click on Information button at the bottom right corner in project drawer, repeat steps above.

Deployed environment differs in two places: it works from Gemfile.lock only, and doesn’t allow any further modifications of Gemfile. The second difference is in bundle directory: we can store bundles in shared directory to save deployment time, it doesn’t change too often anyways. Of course, if you don’t concur, you can always use vendor dir.

This is still a work in progress. Any ideas, comments are highly appreciated.

However, I’m older now, and I tried to find the problem in my own code instead, and I started to rethink what I really want from deployment, and how can I restructure the procedure alongside my new plans.

For now, my target is just a simple app, with no sidekicks like memcached, mongodb, or solr (I definitely plan to have resque), to only one server. It runs on ruby 1.9, in FreeBSD, served by Nginx + Unicorn.  I use ssh+git for repository, and I don’t want to track deployment versions just another way. This is what I ended up with:

• I’ll keep using inploy. It does it’s job, and it can be extended very easily. However, my original inploy config sucks, and it pulls the wrong triggers.
• Deployment tracking is required for staging and production servers, but not on devel servers (yes, I deploy to development servers too, the network is just too baroque for a presentation). The best way to handle them is with git tags. However, inploy doesn’t support them by default.
• Inploy’s rails3 support is a bit rough. It installs packages to production server you normally use in development or test environments, but you don’t need them in production. It bundles gems to .bundle subdir, but this is a feature rvm solves in a more elegant way.
• Sometimes it’s required to have different configurations for different deployment environments. In these situations, copy_sample_files is not enough, but rails3 has great feature called generators, just for this.
• Inploy doesn’t support god (bluepill, monit, etc.), it has to be implemented.

In more general subject, I made the following observations:

• FreeBSD issue: ruby1.9 native gems don’t compile, because of the operating system’s default compiler settings. Ruby Version Manager doesn’t have this problem. Resolution: use rvm.
• Another FreeBSD issue: bash from ports doesn’t run .bashrc when you want to run commands remotely via ssh. Linux doesn’t have this shortcoming, and this feature is explicitely in the bash man page even in FBSD. Resolution: use zsh (use Linux is not an option).
• Regression of rvm: using a global god installation is not an option. Monit and Bluepill have not been considered, because they require root access. Resolution: every application runs it’s own god instance, with the user’s own ruby.
• Multiple application deployments in the same box may cause socket naming clashes in god. Resolution: every god instance uses different socket names.

Enough idle talk, jump right in! I’ll go through them, not necessary in the order I’d actually done it, but in the order you can follow the whole procedure.

First off, you’ll require git and rvm, with the necessary things added, like ruby version, and bundler installed to the global gemset (global gemset is available in all other gemsets). For the latter:

$ bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )
$ echo '[[ -s $HOME/.rvm/scripts/rvm ]] && source $HOME/.rvm/scripts/rvm' >> .rvmstart
$ cat .rvmstart >> .zshenv
$ cat .rvmstart >> .bashrc
$ rm .rvmstart
$ source .rvm/scripts/rvm
$ rvm install 1.9.2
$ rvm use 1.9.2 --default
$ rvm gemset use global
$ gem install bundler --pre

Continue with inploy config:

You can see the config uses special server and template setting, as well as a before_restarting_server block for all environments. There’s another interesting option, deploy.tag: I manage deployments by tags, and it’s dead simple to revert back to a previous release: just re-deploy the previous, proven tag. Of course, this easiness hides it’s traps like database version tracking, but it’s not something we could not solve in finite time. Here comes rails3tags:

Please note setting RAILS_ENV for rake invocation. It is needed because of the tricky bundle installation, which omits test and development groups (and by default it’d start in development mode).

God manager is lightweight too, but it’s not a one-liner either:

It sets up the monitor instance cleanly if it’s not yet set up, (re)loads all configuration, and restarts the whole task. Not more, not less.

OK, let’s move forward. Create a new generator:

$ rails g generator APPNAME/configurations

Then, if you still run Rails3.0.0 beta4, or the bug has not been resolved yet, move lib/generators/configurations into lib/generators/APPNAME directory. The generator itself is not a big deal:

I toyed with the idea of adding a .rvmrc template here, but it is required for first deployment, before any rake tasks can run. Therefore, just put your .rvmrc file in your application root:

That’s it! You can test it out:

$ rake inploy:remote:setup environment=staging
$ rake inploy:up environment=staging

There’s one problem though, for me, an inploy:up runs a couple of times, as if some of the rake tasks would be restarted. I’ll figure it out soon, and I’ll have this post updated.

First issue: ruby and rubygems

Bundler is the new way to handle your gem dependencies. In the extreme it means you don’t have to have any other gem installed than bundler. On the other side, ruby 1.9 is bundled (sic!) with rubygems 1.3.1, but the latest bundler requires rubygems >= 1.3.5. As a side note, installing rails 3.0.0.beta requires rubygems 1.3.5 too, because earlier ones didn’t recognize alphabetic characters in version names. Therefore 1.3.5 is a must.

However, if you update to 1.3.5, you’ll get errors like ‘WARNING: Invalid .gemspec format in …’, and rails command will report other errors too.

If you have a look of irb, you can find the culprit:

This (…/lib/ruby/<version>/rubygems*) points to the bundled (v1.3.1) ruby, while the latest one is installed to …/lib/ruby/site-ruby/<version>/rubygems*. If you have rubygems 1.3.5 installed (gem update –system), you can remove …/lib/ruby/<version>/rubygems* by hand, and the problem is gone.

Second issue: shallow nesting

It seems rails 3.0.0.beta doesn’t have support for shallow nesting. Or at least the new native implementation doesn’t contain it. The funniest part is it seems nobody cares about it. I use shallow nesting myself, so this is a showstopper for me. Rails 3′s router lives in actionpack gem, in ActiveDispatch namespace. Nesting is done by recursive scope changes. The easiest way for manage shallowing the path is to record scope info before scoping, and remove that temporarily from the scope when shallow paths are needed.

Then, all you have to do is to save the scope in front of resources (keep the root of scope shallowing in @scope[:shallow]), and embed block and member route generation calls in ‘with_shallow_scope(saved_scope)’ calls.

Third problem, phasing out gems

I’m using cucumber, rspec, factory_girl for testing, and, most importantly, lockdown for authorization. For now rspec (especially rspec-rails) is not quite there, a lot of stuff has to be rewritten. There is a 2.0.0 alpha prerelease, but it’s really an alpha.

Factory_girl is in the same boat, while the latest github version can be configured somehow, we still miss a gem for that.

No problem, we’ll work with no tests for a while (and/or we won’t actually run the tests). However, user authorization hurts us: lockdown won’t support rails3.

While I think there’re a lot of great stuff out there, I couldn’t yet find anything which would fit my needs. So far we have a lot of great gems out there with excellent DSL’s (CanCan looks pretty neat!), however they all try to solve the problem in a wrong place.

We got a cartload of hints about using Rack for this stuff. Why don’t we? Is it really necessary to pollute all of your controllers? Haven’t you realized that the only place you have to identify authorization is just after routing? Well yes, you can specify that you should be able to do this and that, or you can allow people looking at a page, but they get different response according to their rights. I’m not even convinced that you don’t violate the REST principle by this, but it should not be a big deal adding this to the DSL.

Therefore this is my open question: how can I replace authorization in Rails3?

UPDATE: I have made some changes in shallow routing method to keep paths before shallow.

First, we have to install 2dc_jqgrid, and install it, according to 2dconcept’s document. I didn’t want to install squirrel, because I already added searchlogic, but since squirrel has will_paginate’s functionality built in, we have to deal with it ourselves.

Then, let’s pick an already written CRUD, let’s call ‘users’, which is tested from the UI side (eg. have a test with Webrat / Selenium / anything similar). This is important: making an unobtrusive page needs testable functionality.

First, let’s replace the original table with JqGrid:

index.html template (this is haml, but I think it’s readable):

%table#users
  %tr
    %th= User.human_attribute_name(:username)
    %th= User.human_attribute_name(:email)
    %th= User.human_attribute_name(:tel)
  - @users.each do |user|
    %tr
      %td= h(user.username)
      %td= h(user.email)
      %td= h(user.tel)
      %td
        = link_to t(:show), user
        = link_to t(:edit), edit_user_path(user)
  %tr
    %td{:cols => "5"}= will_paginate
:ruby
  haml_concat jqgrid(t(".title"), "asdf", "/users",
    [
      { :field => "id", :label => "ID", :width => 35, :resizable => false },
      { :field => "username", :label => User.human_attribute_name(:username), :width => 100 },
      { :field => "email", :label => User.human_attribute_name(:email), :width => 140 },
      { :field => "company", :label => User.human_attribute_name(:company), :width => 140 },
      { :field => "tel", :label => User.human_attribute_name(:tel), :width => 80, :align => "center" } ],
    { :height => 220, :selection_handler => "users_select_row", :direct_selection => true })
%p#newuserlink= link_to t(".new"), new_user_path
#userform.popup

It seems height setting was needed because of some browsers. Selection_handler and direct_selection setting are for future extensions. <div id=”userform” class=”popup”>…</div> will be your popup edit box.

So far so good, let’s give JqGrid some data:

UsersController#index:

  def index
    @users = User
    @@index_columns ||= [:id, :username, :email, :company, :telephone, :mobile]

    if params[:_search].present?
      if params[:_search] == "true"
        filters = {}
        @@index_columns.each do |param|
          filters[("#{param}_like").to_sym] = "%#{params[param]}%" if params[param].present?
        end
        @users = User.search(filters)
      end
      if params[:sidx].present?
        @users = if params[:sord] == "desc"
          @users.public_send("descend_by_#{params[:sidx]}")
        else
          @users.public_send("ascend_by_#{params[:sidx]}")
        end
      end
    end
    @users = @users.paginate(:page => params[:page], :per_page => params[:rows] || 10)

    respond_to do |format|
      format.html
      format.json do
        render :json => @users.to_jqgrid_json(
          @@index_columns, params[:page], params[:rows], @users.total_entries
        )
      end
    end
  end

As you can see, we kept html output as is, but we added extra functionality jqgrid search and filter. It’s a bit more verbose than with Squirrel, but that’s alright. Now your page should turn the original table to a grid if you have javascript enabled. Let’s handle our first action: open an in-page popup for editing an entry:

index.html template, annotated:

For first, tell jQuery to ask for Javascript responses to ajax requests:

jQuery.ajaxSetup({
       'beforeSend': function(xhr) {xhr.setRequestHeader("Accept", "text/javascript")}
})

Then, register events when the page is loaded. Start with New User link in the bottom. Note that inject_userform_submit is called after the response is arrived: we can’t register the edit form’s submit event until it’s loaded, we have to inject the code after we get it loaded.

    $(function() {
      $("#newuserlink").click(function() {
        $.get("/users/new", null, inject_userform_submit, "script");
        return false;
      })

This is the meat of our popup: I use jQuery Tools for that.

      $(".popup").overlay({
        expose: {
          color: '#fff',
          loadSpeed: 100,
          opacity: 0.5
        }
      })
    })

Let’s handle JqGrid’s row selection callback too, just like #new:

    users_select_row = function(id) {
      $.get("/users/"+id+"/edit", null, inject_userform_submit, "script")
    }

Last but not least replace the popup edit/new form’s submit to handle things in Javascript, and make this popup opened:

    inject_userform_submit = function() {
      $("#userform form").submit(function() {
        $.post($(this).attr("action"), $(this).serialize(), null, "script");
        return false;
      })
      $(".popup").overlay().load()
    }

Now we have requests, let’s create responses. I really hope everybody puts the form genrator in ‘form’ partial, and we can reuse it:

new.js.erb:

$("#userform").html("<%= escape_javascript(render 'form') %>")

edit.js.erb:

$("#userform").html("<%= escape_javascript(render 'form') %>")

create.js.erb:

<% if flash.has_key?(:notice) %>
  $("#flashes").html('<div class="flash notice"><%= escape_javascript(flash.delete(:notice)) %></div>')
  $("#users").trigger("reloadGrid")
  $(".popup").overlay().close()
<% else %>
  $("#flashes").html('<div class="flash error"><%= escape_javascript(flash.delete(:error)) %></div>')
  $("#userform").html("<%= escape_javascript(render 'form') %>")
<% end %>

You can make it more elegant if you have another flash div if you re-render the form.

Then, we have to get UsersController#create and UsersController#update behave differently:

UsersController#create:

    ...
    if @user.save
      flash[:notice] = t('users.create.success')
      respond_to do |format|
        format.html { redirect_to users_path }
        format.js
      end
    else
      flash.now[:error] = t('users.create.failure')
      respond_to do |format|
        format.html { render "new" }
        format.js
      end
    end
    ...

UsersController#update:

  def update
    @user = User.find(params[:id])
    if @user.update_attributes(params[:user])
      flash[:notice] = t('users.update.success')
      respond_to do |format|
        format.html { redirect_to users_path }
        format.js { render "create" }
      end
    else
      respond_to do |format|
        format.html { render "edit" }
        format.js { render "create" }
      end
    end
  end

Then, of course, you have to do some CSS magic to make it nice:

.popup {
  width: 408px;
  display: none;
  border: 2px solid #666;
  background-color: #324; }

Or, in SASS:

.popup
  :width 408px
  :display none
  :border 2px solid #666
  :background-color #324

We have achieved our goals:

• the original code is still working (run the tests again, you can measure this)
• the previous one vice-versa: our tests are still greens
• we used our old gems (searchlogic, will_paginate)
• the code is unobtrusive, everything works fine if JavaScript is not enabled

Enjoy!

However, a couple of weeks ago, I noticed that Lockdown reached 1.0.  Moreover, now it has Authlogic support, and even model-level permissions!  Yikes!  Of course fighting lockdown not to lock down everything was not easy, especially in the tests.  I think it is worth to have my experiences documented.I don’t want to bother you with basic installation.  You can find excellent documentation about it both in Authlogic and Lockdown pages. Go read them.

However, it’s important to do some other changes, especially if you want to use model-level permssions. Most likely you have a User and a UserGroup model. It’s wise to connect them with has_and_belongs_to_many in both directions.  Therefore you can ask all groups for a user, and all users for a group with ease.  You can exploit this for granting multiple levels of access:

In Lockdown, you cannot write a permission like this:

set_permission(:user_admin).with_controller(:users)
set_permission(:user_owner).with_controller(:users).
    only_methods(:new, :create).
  and_controller(:users).
    only_methods(:edit, :update).
      to_model(:user).where(:id).equals(:current_user_id)
set_protected_access :users_owner
set_group_access(:usersadmin, :user_admin)

The problem is if you want to do your stuff as user_admin, restrictions from user_owner creep in, and you won’t be able to update other users’ data. Therefore, you have to implement this partly in the model, optimizing out intersecting permission settings:

set_permission(:user_admin).with_controller(:users).
  only_methods(:new, :create)
set_permission(:user_owner).with_controller(:users).
  only_methods(:edit, :update).
    to_model(:user).where(:editable_by).includes(:current_user_id)
set_protected_access :users_owner
set_group_access(:usersadmin, :user_admin)

Then, let’s implement editable_by in the model:

app/models/user.rb:

def editable_by
  @editable_by_ids ||= UserGroup.find_by_name("Usersadmin").user_ids | [id]
end

It’s simple, ain’t it? So far so good, but we haven’t start to write specs (or tests, YMMW). Don’t hesitate.

You have to have both modules set up properly for testing.  Let’s write some support code for both (you can fnd a good example for unit tests in lockdown-rails-app):

spec/support/rbac.rb:

require 'authlogic/test_case'
def login_user(group = nil)
  initialize_user(group)
  create_user_session
end

def initialize_user(group)
  @loginuser = Factory.build(:user)
  @loginuser.stub!(:save).and_return(true)
  @loginuser.stub!(:id).and_return(2)
  User.stub!(:find).and_return(@loginuser)
  return if group.nil?
  set_user_group(group)
end

def set_user_group(group)
  ug = Factory(:user_group, :name => group)
  @loginuser.stub!(:user_groups).and_return([ug])
end

def create_user_session
  activate_authlogic
  @current_user_session = UserSession.create(@loginuser)
  controller.send :add_lockdown_session_values, @loginuser
end

We could write more, but let’s face it, we’re working with real life groups, they should be available. The long story short, all you have to do is to call login_user if you want to manipulate data as a logged in user.  Call login_user("usergroup") if you wan to have group membership. Plain and simple.

Then, all you have to touch is your controller specs (or any test which uses a controller), beside editable_by methods.  It’s wise to group your tests for guest, normal user, and group member access.  Generally speaking, you can set up normal user tests like

before(:each) do
  login_user
  @loginuser.stub!(:editable_by).and_return([@loginuser.id])
  subject.should_not receive(:access_denied)
end

However, be prepared for extra model access because of model-level permissions!  It’s safe to say every model reference will be resolved for the model queried, and if you already have a mock for the query, you have to specify the number of calls:

User.should_receive(:find).twice.and_return(@loginuser.id)

Apart from these changes, you can develop with Lockdown very easily.

I still haven’t fixed my cucumber features yet, but I plan to have this solution there also, letting me write all my Given’s in direct model access.