|
|
|
|
Bookmark this on Delicious
Share on Facebook
 Slashdot It!
Digg flock
Portable advisory file locking
(PHP 4, PHP 5)
flock() allows you to perform a simple reader/writer model which can be used on virtually every platform (including most Unix derivatives and even Windows). The lock is released also by fclose() (which is also called automatically when script finished). PHP supports a portable way of locking complete files in an advisory way (which means all accessing programs have to use the same way of locking or it will not work). Parameters
ChangeLog
ExamplesExample 638. flock() example<?phpNotes
Note:
flock() locks mandatory under Windows.
Note:
Because flock() requires a file pointer, you may have to use a special lock file to protect access to a file that you intend to truncate by opening it in write mode (with a "w" or "w+" argument to fopen()).
Warning:
flock() will not work on NFS and many other networked file systems. Check your operating system documentation for more details. On some operating systems flock() is implemented at the process level. When using a multithreaded server API like ISAPI you may not be able to rely on flock() to protect files against other PHP scripts running in parallel threads of the same server instance!
flock() is not supported on antiquated filesystems like
Related Examples ( Source code ) » flock Examples ( Source code ) » Integer Arguments to the flock() Function Examples ( Source code ) » Write content to file Code Examples / Notes » flockcarl
[editors note: try out php.net/dio_fcntl] As flock() doesn't work over NFS (whereas fcntl() does, but there's no PHP interface for that), you may have to provide some sort of backup locking system if for one reason or other you need to use NFS (or VFAT, but as the documentation points out it's antiquated). One way of achieving this is to use a relational database. If you have a table T with an integer column cnt and a _single_ row, you can: UPDATE T set cnt = cnt + 1 WHERE cnt >= 0 to obtain a shared lock (and of course you need to verify that there was one affected row). For an exclusive lock: UPDATE T set cnt = -1 WHERE cnt = 0 This scheme has two problems: You rely on the script to be able to release locks before exiting, and there is no way to wait for a lock (except polling). On the whole, if you can use flock() instead, use flock(). lockwang
your code has a mistake , i think you should change[ $lf = @fopen ($lf, 'x');] to [ $lf = @fopen ($lock, 'x');] and your code has a big problem that if your lockfile is not unlinked for any reason and your function will not work properly. so i think your function is not a good one. dranger
You should never have to write code like this: <?php while(!($canWrite=flock($f, LOCK_EX))) { usleep(2000); } // go ahead and use file fwrite(...); fclose($f); ?> This absolutely defeats the purpose of OS-supported locks, and what's worse is if you specify LOCK_NB, it eats up your CPU time. The following code will work the same way: <?php // this will automatically block, that is, PAUSE until the lock is released flock($f, LOCK_EX) or die("Error getting lock!"); // go ahead and use file fwrite(...); fclose($f); ?> I'm not sure what these people creating spin/sleep loops are doing. geoff
You should lock the file _before_ opening it, and unlock _after_ closing it. Othervise an another process might be able to access the file between the lock/unlock and the open/close operation.
eddi.to-grip.de
When using a multithreaded server like apache with worker.c on Linux you can use follow lines: <?php function fprotect($key, $file, $content=FALSE, $fopen_mode='w', $chmod=0600, $sem_mode=0600 ) { $semaphor=sem_get($key,1,$sem_mode); sem_acquire($semaphor); if($content) { if($dat=fopen($file,$fopen_mode)) { chmod($file,$chmod); fputs($dat,$content); fclose($dat); } else return(FALSE); } else return(file_get_contents($datei)); sem_release($semaphor); return(TRUE); } ?> Also this should work instead of flock() using a non-multithreaded server on Linux. Greetings form Berlin :) relsqui
Well, I can claim I wrote this as a workaround for earlier versions of PHP, but in reality I just didn't think to look up flock() until after I'd written it. However, if you do happen to have an earlier version of PHP, these might come in handy. Enjoy. <?php // Lock a file, timing out if it takes too long. function lock ($lock, $tries) { $lock0 = ".{$lock}0"; $lock1 = ".{$lock}1"; for ($i=0; $i<$tries; $i++) { if (!is_file($lock0)) { touch($lock0); if (!is_file($lock1)) { touch($lock1); return 1; } } usleep(100); } return 0; } // Unlock a file. function unlock ($lock) { unlink(".{$lock}1"); unlink(".{$lock}0"); } // Usage example. $filename = "somefile"; $data = "stuff and things\n"; $tries = 10; if (lock($filename, $tries)) { $h = fopen($filename, "a") or die(); fwrite($h, $data); fclose($h); unlock($filename); } else { die("Failed to lock $filename after " . ($tries*100) . " milliseconds!"; } ?> sbsaylors
Well I needed some kind of "lock" in windows, so I just created a lock.file in that directory. So if anything wants to 'write' to that file (or append, whatever) it checks to see if a lock.file (or whatever name you want to use) is there and if the date of the file is old. If its old (1 hour) then it deletes it and goes ahead. If its not old then it waits a random time. When its finised it deletes the file. Seems to work alright but I havent tested anything and its only used for maybe a 100 hits a day but... :) 04-sep-2002 07:58
Warning! When PHP is running as a CGI or fast-CGI extension in Apache, flock() does not guarantee that your file updates will complete!!! If a browser starts a request and interrupts it fast enough by using many F5-key refreshes, Apache will KILL the PHP process in the middle of the update operation (because it will detect an unexpected socket close before the PHP script is complete), leaving an incomplete or truncated file! A simple PHP script that increments a counter in a text file will demonstrate this: to update the counter, one needs to gain a lock then open the counter file in "a+" mode, rewind it, read it, rewind again, ftruncate it befire writing the new value and closing the counter file and the lock file. Have your script display the new counter value. Now use your favorite browser on your script, and make many refreshes from the same PC, using the F5-Refresh key, you'll see that sometimes the counter returns to 0, because there's an interruption after the counter file was truncated but before the new counter value was written to the file! Note that this affects also simple databases updates without rollback logs such as MySQL! Before updating any data, as a security measure, and if you don't have rollback logs, you should strictly limit the number of update requests per seconds a user can make through your PHP script. Shame, this requires a database or logging file to enable tracking user activity. The only solution is to use an auto-backup system (for file-based databases), or a database engine with rollback capability... I don't know if PHP implements a way to handle gracefully kill signals sent by Apache, so that we can ensure that we can complete a critical update operation. For this problem, flock() is not a solution! This is a MAJOR security issue for all sites that use text-file based databases! bwo
Using a secondary file for locking isn't too bad. You simply surround your actual writes by the flock commands on the locking file. Because of the blocking, your writes will not execute until you get the exclusive lock. <?php $filename = "data.txt"; $lock = ".lock"; // Lock the .lock file. $lock_fp = fopen($lock, "r"); if (!flock($lock_fp, LOCK_EX)) { echo "locking failed"; } // Write to the data file. $fp = fopen($filename, "w"); fwrite( $fp, "word up!"); fclose($fp); // Release the lock. flock($lock_fp, LOCK_UN); ?> Easy as pi. hhw
Users may find that the following is more useful than flock. function filelock($operation) { $lock_file = "filelock.txt"; switch($operation) { case LOCK_SH: // 1 case LOCK_EX: // 2 do { clearstatcache(); usleep(rand(5,70)); } while(file_exists($lock_file)); $fp = fopen2($lock_file, "wb"); fwrite($fp, "blah blah blah"); fclose($fp); break; case LOCK_UN: // 3 do { clearstatcache(); usleep(rand(5,70)); } while(!file_exists($lock_file)); unlink($lock_file); break; default: break; } } mphh
To solve most of the "PHP quit writing my file and messed it up" problems just add: ignore_user_abort(true); before fopen and: ignore_user_abort(false); after fclose, now that was easy. rehfeld.us
This is based on the function by jeroenl at see dot below and the suggestion of jeppe at bundsgaard dot net please note this wont work on windoes unless using php5, due to the use of usleep() one may wish to tailor some of the values to better suit thier application. the rand() values, the number of loop attempts, and the max age of the lockfile are what im referring to <?php function locked_filewrite($filename, $data) { ignore_user_abort(1); $lockfile = $filename . '.lock'; // if a lockfile already exists, but it is more than 5 seconds old, // we assume the last process failed to delete it // this would be very rare, but possible and must be accounted for if (file_exists($lockfile)) { if (time() - filemtime($lockfile) > 5) unlink($lockfile); } $lock_ex = @fopen($lockfile, 'x'); for ($i=0; ($lock_ex === false) && ($i < 20); $i++) { clearstatcache(); usleep(rand(9, 999)); $lock_ex = @fopen($lockfile, 'x'); } $success = false; if ($lock_ex !== false) { $fp = @fopen($filename, 'a'); if (@fwrite($fp, $data)) $success = true; @fclose($fp); fclose($lock_ex); unlink($lockfile); } ignore_user_abort(0); return $success; } ?> derf
This addresses a few topics posted earlier: First (to sbsaylors), using a seperate lockfile has problems since you must seperately check the lockfile, then create the lockfile, then perform your action. In the event of two processes running at the same time, you may wind up in the situation where there is no file and execution proceeds as follows: A:checks-for-file B:checks-for-file A:creates-file B:creates-file A:open-file B:open-file A:writes B:writes ... now, both A and B have written different things to the file, which in most systems would be buffered in seperate memory for A and B. Thus, if A closes first, when B closes its buffer will overwrite A. If B closes first, A will overwrite B. Other situations may arise where everything looks ok, because B's entire execution happens before A opens the file. But then, B will delete the lock file and A will be unprotected. Second (to the nameless person): flock doesn't exist to guarantee that your program doesn't stop running. It only makes sure that other programs that bother to flock() won't mess with the file while you're messing with it. Proper method to do this would be to write into a temporary file, then copy that file over the original in a single operation. If the execution stops, only the temp file is corrupt. llinear
There are few instances where several instances of a server script need write access to the same file. But it is not mission critical (you would not be using PHP) that the write succeeds, and therefore not worth making a script wait. If its just for a counter, then you can afford to skip a client from time to time... Bots already screwed your count
antti haapala
The supplied documentation is vague, ambiguous and lacking, and the user comments contain erroneous information! The flock function follows the semantics of the Unix system call bearing the same name. Flock utilizes ADVISORY locking only; that is, other processes may ignore the lock completely; it only affects those that call the flock call. LOCK_SH means SHARED LOCK. Any number of processes MAY HAVE A SHARED LOCK simultaneously. It is commonly called a reader lock. LOCK_EX means EXCLUSIVE LOCK. Only a single process may possess an exclusive lock to a given file at a time. If the file has been LOCKED with LOCK_SH in another process, flock with LOCK_SH will SUCCEED. flock with LOCK_EX will BLOCK UNTIL ALL READER LOCKS HAVE BEEN RELEASED. If the file has been locked with LOCK_EX in another process, the CALL WILL BLOCK UNTIL ALL OTHER LOCKS have been released. If however, you call flock on a file on which you possess the lock, it will try to change it. So: flock(LOCK_EX) followed by flock(LOCK_SH) will get you a SHARED lock, not "read-write" lock. majer
The function flock() I can use only after I open file for blocking writing or reading to file. But I found, that function flock() do not block file agains delete by $f=fopen($file,"w"); nor agains ftruncate($f,0); sorry my English administrator
The bug fix in my last post. During test I have discovered that if you have locked file for writing (LOCK_EX) both read and write will not be accessible from other scripts. In case of read locking (LOCK_SH) only writing will not be accessible for other PHP scripts but they will be able to read file simultaneously.
name
regarding the first post, microsleep() will sleep for the specified number of microseconds dranger
Re: Niels Jaeckel The code posted unfortunately does NOT work in race conditions, because even though it is improbable, there is still a chance the 'mutex' will fail and you will encounter a situation where two programs have a 'lock' for the same file at the same time. It is improbable, but in a busy system with perhaps many processes trying to open one file or perhaps two processes opening the file several times, the probability rises and you will most certianly get strange bugs at strange times. Even though it looks quite improbable, there are two problems: 1) The extra ifs do not add any extra precautions. You might as well usleep for a few microseconds because only the last one matters. 2) Even though it looks like a single atomic command, if you look at the PHP source code, touch is relatively complicated. It gets the time from the computer, checks the base directory, etc.,etc. This means that there will be times that this lock fails, no matter how improbable. What makes a race condition so nasty is that it happens only once every so often, making it nearly impossible to debug and really annoying. *** The only safe way to implement locks is with flock or some other locking mechanism *outside* of PHP. *** (caveat: there may be an alternate locking mechanism in PHP i don't know about, but you cannot make your own) To put this to rest, make a PHP script using the functions described in the previous post. Then, add this code to the file: <?php /*** test.php ***/ // lock and unlock function defs go here while(1) { if(lock("test")) { $f=fopen("importantfile", "w") or die; $pid=getmypid(); $string="Important Information! From $pid"; fwrite($f, $string); fclose($f); $check=file_get_contents("importantfile"); if($check != $string) { echo "THIS LOCK FAILED!\\n"; } unlock("test"); } } ?> Then run this script from the command line - it will loop forever happily. Then, *while that script is running*, run the same script again while the first one is still going. In a UNIX environment you can do this by typing: > php test.php & > php test.php & You will probably see this: Warning: unlink(test.lock): No such file or directory in lock.php on line 29 THIS LOCK FAILED! Warning: touch(): Utime failed: No such file or directory in lock.php on line 19 a lot of times. This means the lock has failed :) In fact, if you ever think you have invented a clever way to lock a file, test it first in this while loop. Just replace lock and unlock with your function and rearrange the code so it makes sense. Then run it and see if it fails. Note that flock() passes this test beautifully. jbr
On Windows ISAPI testing a file or directory with flock does not return TRUE OR FALSE, it just results in a system hang until the prior flock called has been released. So it is sometimes better to create a fake locking system that gives you complete control of the IO handle so you can exist or move on if the lock is not released at a certain number of tries or time, to get at the file! // PHP 5 example <? $file = './log.txt'; // the file we are writing to $data = "add this\r\n"; // data to add $write_type = 'a'; // append to the file $lock_name = './lock'; // the fake locking file name $lock_tries = 10; // the number of times to try and open the file // usage if ( fake_lock ( $file, $data, $write_type, $lock_name, $lock_tries ) === true ) { echo 'data written to file'; } else { echo 'could not write data to file!'; } function fake_lock ( $fl_file, $fl_data, $fl_write, $fl_name, $fl_try ) { $fl_done = false; $fl_count = 0; $fl_key = md5 ( uniqid ( microtime () ) ); do { $fl_count++; if ( ! file_exists ( $fl_name ) ) { file_put_contents ( $fl_name, $fl_key ); $fl_test = file_get_contents ( $fl_name ); if ( $fl_test == $fl_key ) { $fl_done = true; } } if ( ! $fl_done ) { if ( $fl_count == $fl_try ) { return ( false ); } else { usleep ( 10000 ); } } } while ( ! $fl_done ); $io = fopen ( $fl_file, $fl_write ); fputs ( $io, $fl_data ); fclose ( $io ); unlink ( $fl_name ); return ( true ); } ?> jeroenl
lockwang, thanx for pointing out the bug I forgot to update overhere. I agree (as mentioned) that an old not unlinked lockfile will stop writing, but at least writing will never corrupt old data and if needed you can add a lock clean up. If you want uncorrupted files and a possible data loss is not a very big problem (like with traffic logging, etc.) this solution does work. After trying a lot of things (always corrupting files) I use this code now since a few months for loggings and counters on many sites, resulting in (gzipped) files up to 2 Mb, without any corrcupted file or any not unlinked lockfile yet (knock knock...). A better solution (100% save would be nice :) would be great, but since I haven't seen one around yet ... If more is needed SQLite (if available) might be an alternative. Corrected code: <? function fileWrite($file, &$str, $lockfile = null) { // ceate a lock file - if not possible someone else is active so wait (a while) $lock = ($lockfile)? $lockfile : $file.'.lock'; $lf = @fopen ($lock, 'x'); while ($lf === FALSE && $i++ < 20) { clearstatcache(); usleep(rand(5,85)); $lf = @fopen ($lock, 'x'); } // if lockfile (finally) has been created, file is ours, else give up ... if ($lf !== False) { $fp = fopen( $file, 'a'); fputs( $fp, $str); // or use a callback fclose( $fp); // and unlock fclose($lf); unlink($lock); } } ?> rob
locking a file exclusively *and* in non-blocking mode: if (flock($fh, LOCK_EX | LOCK_NB)) { // do sumthin } else { // fail, report file locked } rudy dot metzger
Like a user already noted, most Linux kernels (at least the Redhat ones) will return false, even if you locked the file. This is because the lock is only ADVISORY (you can check that in /proc/locks). What you have to do there is to evalute the 3rd parameter of flock(), $eWouldBlock. See for an example below. Note however that if you lock the file in non blocking mode, flock() will work as expected (and blocks the script). <?php $fp = fopen( "/var/lock/process.pid", "a" ); if ( !$fp || !flock($fp,LOCK_EX|LOCK_NB,$eWouldBlock) || $eWouldBlock ) { fputs( STDERR, "Failed to acquire lock!\n" ); exit; } // do your work here fclose( $fp ); unlink( "/var/lock/process.pid" ); ?> damasta
just wanted to say that you will most likely fail if you use a separate lock file together with register_shutdown_function. my script did some different actions... resizing pictures, rotating them and this stuff. it needed a "database" file to get the correct file locations. this database file also stored some flags. and of course the script had to save that file when it was done. because of my script exited on many different points depending on the action i used register_shutdown_function to save the file. it wanted to use a locking system to be sure the script doesn't overwrite the data another process had written into it some microseconds before. i was running on windows 2000 and apache2 on my developing machine, and flock always returned true for some reason... so i used a separate lock file. the script looked for it at the beginning and exited if it was found. otherwise it created it. but this file had to be deleted at the end. i put the unlink command into the registered shutdown-function but it never deleted the file. i tried clearstatcache and some other stuff but it didn't help. maybe this helps someone. dranger
jbr at ya-right dot com has the wrong idea of what flock is supposed to do. flock is *supposed* to "hang" until the other lock is released. This is the intended behavior on ALL systems, not just Windows. If this behavior is not acceptable, flock already has a mechanism to deal with this: LOCK_NB. Just add LOCK_NB to the second argument, and flock will not "hang." For example: <?php flock($fp, LOCK_EX+LOCK_NB, $wouldblock) ?> and flock will return TRUE or FALSE immediately, setting $wouldblock to 1 if the call would have blocked. You DO NOT need to muck about with while loops, usleep, "clever" md5 + time hashing tricks, random numbers, or any of this nonsense. These techniques defeat the entire point of using flock in the first place. What's worse about jbr's code is that it introduces a bug that flock was meant to fix in the first place! It's called a "race condition" - here's an example: 1. Program A checks that file "fakelock" exists. It doesn't. 2. Processor switches to Program B. 3. Program B checks that file "fakelock" exists. It doesn't. 4. Program A writes its unique key, "A" to the file, and checks that its key is correct. It is. 5. Program B writes its unique key "B" to the file, and because its file pointer was pointing at the beginning of the file, it overwrites "A" with "B". It checks that its key is correct. It is. 6. Now Program A and Program B can both write to file "$fl_file.", and both overwrite the file that was to be protected. There are many other situations in which this code will fail as well. To the best of my knowledge, YOU CANNOT MAKE YOUR OWN FLOCK USING JUST PHP NO MATTER HOW HARD YOU TRY. YOU MUST USE FLOCK. If you have Windows ISAPI, you may have trouble with flock working correctly because it is multithreaded, but this kind of fix will not correct the situation. jerry
Indeed, flock() will not work reliably when the underlying filesystem is NFS. The proper way to perform file locking, in this case, would be to use PHP's link() function. From the Linux man page of open(): O_EXCL When used with O_CREAT, if the file already exists it is an error and the open will fail. In this context, a symbolic link exists, regardless of where its points to. O_EXCL is broken on NFS file systems, programs which rely on it for performing lock- ing tasks will contain a race condition. The solution for per- forming atomic file locking using a lockfile is to create a unique file on the same fs (e.g., incorporating hostname and pid), use link(2) to make a link to the lockfile. If link() returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to check if its link count has increased to 2, in which case the lock is also successful. php
In response to the solution below, it should be noted that Semaphore, Shared Memory and IPC Functions (which include sem_get, sem_acquire, and sem_release) are not supported by default in PHP.
corey
In response to korostel: Using files for things like counters is a very bad idea...especially when you're requiring scripts to wait until the file is available before accessing the file. You should use a database like MySQL or PostgreSQL since they are much better performance-wise (especially MySQL). If you need the data to be in a file for some reason you can always use a cron job that reads the current data from your database and then writes it to the file. korostel
In order to prevent access to some counter file we can use another file as a flag instead of flock(). We change flag's file mode to '400' each time we want to change the counter, and set it back to '600' at the end. If the mode of the flag's file was already changed by another process, we make delayed loop till its mode is set to writable again. The page is refreshing every second so the results can be seen by opening it in two or three browser windows. I tested the script meny times and still my counter is safe. Hope this will help. --------------START------------------ <? header("Expires: Mon, 26 Jul 1997 05:00:00 GMT"); header("Cache-Control: no-cache, must-revalidate"); header("Pragma: no-cache"); header("Last-Modified: ".Date("D, d M Y H:i:s")." GMT"); define('COUNT_FILE','count.txt'); //Our counter file define('LOCK_FILE','lock.txt'); //Our lock file. function read_write () { //making our lock file nonwritable chmod(LOCK_FILE, 0400); //reading clearstatcache(); $fpr = fopen(COUNT_FILE, "r"); $count=fread ($fpr, filesize (COUNT_FILE)); fclose($fpr); $count++; //writing $fpw = fopen(COUNT_FILE, "w"); fwrite($fpw,$count); fclose($fpw); //pause the script just to see how it works sleep(1); //making our lock file writable again chmod(LOCK_FILE, 0600); return $count; } ?> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html><head><title>Untitled</title><meta http-equiv="Refresh" content="1"></head><body> <? clearstatcache(); //Check the mode of our lock file if(is_writable(LOCK_FILE)) { print 'is unlocked '; //make changes and show the results print 'changed to '.read_write().' '; }else{ print 'is locked: '; //make loop while(!is_writable(LOCK_FILE)) { print 'pause... '; usleep(rand(5,999)); } //make changes and show the results print ' changed at last to '.read_write().' '; }?> </body></html> --------------END------------------ 12-jan-2006 10:51
If you use a database, you can also create timeout locks.
glip
If you don't want use secondary lock file while truncating, try this: <?php $fp = fopen("/tmp/lock.txt", "a+"); if (flock($fp, LOCK_EX)) { // do an exclusive lock ftruncate($fp, 0); fwrite($fp, "Write something here\n"); flock($fp, LOCK_UN); // release the lock } else { echo "Couldn't lock the file !"; } fclose($fp); ?> georgeskesseler
If you do some testing opening several tabs in firefox on the same URL you may find that LOCK_NB seems not to get respected and $wouldblock always is 0. Apparently this is a firefox problem. Try 2 different browsers or 2 separate wget, then there is no problem. http://bugs.php.net/bug.php?id=36824 I also confirm that you have to test the value of $wouldblock, not the return value of flock() if using LOCK_NB on linux (debian/ubuntu 2.6.15-26-386) info
If there is a file that´s excessively being rewritten by many different users, you´ll note that two almost-simultaneously accesses on that file could interfere with each other. For example if there´s a chat history containing only the last 25 chat lines. Now adding a line also means deleting the very first one. So while that whole writing is happening, another user might also add a line, reading the file, which, at this point, is incomplete, because it´s just being rewritten. The second user would then rewrite an incomplete file and add its line to it, meaning: you just got yourself some data loss! If flock() was working at all, that might be the key to not let those interferences happen - but flock() mostly won´t work as expected (at least that´s my experience on any linux webserver I´ve tried), and writing own file-locking-functions comes with a lot of possible issues that would finally result in corrupted files. Even though it´s very unlikely, it´s not impossible and has happened to me already. So I came up with another solution for the file-interference-problem: 1. A file that´s to be accessed will first be copied to a temp-file directory and its last filemtime() is being stored in a PHP-variable. The temp-file gets a random filename, ensuring no other process is able to interfere with this particular temp-file. 2. When the temp-file has been changed/rewritten/whatever, there´ll be a check whether the filemtime() of the original file has been changed since we copied it into our temp-directory. 2.1. If filemtime() is still the same, the temp-file will just be renamed/moved to the original filename, ensuring the original file is never in a temporary state - only the complete previous state or the complete new state. 2.2. But if filemtime() has been changed while our PHP-process wanted to change its file, the temp-file will just be deleted and our new PHP-fileclose-function will return a FALSE, enabling whatever called that function to do it again (ie. upto 5 times, until it returns TRUE). These are the functions I´ve written for that purpose: <?php $dir_fileopen = "../AN/INTERNAL/DIRECTORY/fileopen"; function randomid() { return time().substr(md5(microtime()), 0, rand(5, 12)); } function cfopen($filename, $mode, $overwriteanyway = false) { global $dir_fileopen; clearstatcache(); do { $id = md5(randomid(rand(), TRUE)); $tempfilename = $dir_fileopen."/".$id.md5($filename); } while(file_exists($tempfilename)); if (file_exists($filename)) { $newfile = false; copy($filename, $tempfilename); }else{ $newfile = true; } $fp = fopen($tempfilename, $mode); return $fp ? array($fp, $filename, $id, @filemtime($filename), $newfile, $overwriteanyway) : false; } function cfwrite($fp,$string) { return fwrite($fp[0], $string); } function cfclose($fp, $debug = "off") { global $dir_fileopen; $success = fclose($fp[0]); clearstatcache(); $tempfilename = $dir_fileopen."/".$fp[2].md5($fp[1]); if ((@filemtime($fp[1]) == $fp[3]) or ($fp[4]==true and !file_exists($fp[1])) or $fp[5]==true) { rename($tempfilename, $fp[1]); }else{ unlink($tempfilename); if ($debug != "off") echo "While writing, another process accessed $fp[1]. To ensure file-integrity, your changes were rejected."; $success = false; } return $success; } ?> $overwriteanyway, one of the parameters for cfopen(), means: If cfclose() is used and the original file has changed, this script won´t care and still overwrite the original file with the new temp file. Anyway there won´t be any writing-interference between two PHP processes, assuming there can be no absolute simultaneousness between two (or more) processes. fackelkind honorsociety de
If found a handy way for myself, to handle a filelock on linux and windows: <?php function &file_lock (&$handle, $lock = true){ while (false === flock ($handle, ($lock == false) ? LOCK_UN : LOCK_EX + LOCK_NB)) usleep (300); } ?> May it help So far will reiher
I've been testing a few custom file access functions but I always liked the simplicity of file_get_contents(). Of course it doesn't seem to respect any file locks created with flock(). I created the function below to wrap around file_get_contents() so it can support locked files. It's an odd way of doing it but it works for me. function flock_get_contents($filename){ $return = FALSE; if(is_string($filename) && !empty($filename)){ if(is_readable($filename)){ if($handle = @fopen($filename, 'r')){ while(!$return){ if(flock($handle, LOCK_SH)){ if($return = file_get_contents($filename)){ flock($handle, LOCK_UN); } } } fclose($handle); } } } return $return; } joby
I'm thinking that a good way to ensure that no data is lost would be to create a buffer directory that could store the instructions for what is to be written to a file, then whenever the file is decidedly unlocked, a single execution could loop through every file in that directory and apply the indicated changes to the file. I'm working on writing this for a flat-file based database. The way it works is, whenever a command is issued (addline, removeline, editline), the command is stored in a flat file stored in a folder named a shortened version of the filename to be edited and named by the time and a random number. In that file is a standardized set of commands that define what is to be done to what file (the likes of "file: SecuraLog/index_uid" new line "editline: 14"). Each execution will check every folder in that directory for files and a certain amount of time (I don't know how long, maybe 1-2 seconds) is spent making pending changes to unlocked files. This way no changes will be lost (i.e. person 1 makes a change at the same time as person 2, and person 1 loses the race by just enough to have their changed version of the file overwritten by person 2's version) and there will be no problems with opening an empty open file. klaus rabenstein von heilabruck
I'm not quite sure, whether it is possible to create a effective machanism that ensures file locking by creating something called a "lock-file" (Important: this lock-file isn't thought as file with contents, it only has symbolic meaning if it exists). There seem to be several reasons for NOT doing the above: 1. You have to check, whether the lock-file exists 1.1 Do something if it exists 1.2 Do something if it doesn't exist 2. Ensure the lock file has rights that prevent messing up with it from another another program 3. and so on... Hence, it comes forward that this method is very slow and not effective!!! The argument "flock() is no good because of differing platform based handlings" fails because the methods you need to use the above are at least as much dependable on platforms as flock() is itself. I'm not commenting on many posters before me for these reasons. Let's consider you want to write some contents (C) to a file (F). In my opinion a temporay file (T) with a random filename should store the content you want to write to F. After you successfully stored C in T you open a stream to F an close it just after you write C in it. If everything went ok, you can delete T. Otherwise you output a message and encourage the user to try it again. One can use the contents saved in T as optional recovery data. This way it is highly impossible that another program is messing up with your data because T's name can hardly be guessed, and if flock() works on your system, it is locked as well. Of course, my method extends the meaning of flock(), but I do not see another way how to deal with problems that might occur when using flock() only. I hope I wasn't too theoretical. wildcat
I wrote the following function to make a script wait in a queue of instances accessing the same shared resource and then lock it, as a part of a lock-access-release advisory mechanism based on locking directories. It works on my web site (also passes the loop test proposed by dranger below). Might show unpredicted behaviors on other systems, under conditions I didn't think at or produce, expecially when mkdir atomicity is not guaranteed. The whole script is too long for posting, but everyone interested in looking for eventual bugs, testing, torturing and using it, can download code and documentation from http://brightpages.net/doc > QLCK. Hope it is useful. <?php function qLock($lockSuffix, $tmpDir, $maxWait, $padLength) { if ($tmpDir == '' or !is_dir($tmpDir)) $tmpDir = '.'; $maxWait = (int) $maxWait; if ($maxWait <= 0) $maxWait = 60; $padLength = (int) $padLength; if ($padLength <= 0) $padLength = 6; if ($lockSuffix == '') { return false; } else { $currentdir = getcwd(); chdir($tmpDir); // relative to the current directory $n = 0; $lockdirs = glob('*-*.'.$lockSuffix, GLOB_ONLYDIR) or array(); foreach($lockdirs as $lockdir) { list($m, ) = explode('-', $lockdir); $m = intval($m); if ($m > $n) $n = $m + 1; } $npad = str_pad((string) $n, $padLength, '0', STR_PAD_LEFT); while (glob($npad.'-*.'.$lockSuffix, GLOB_ONLYDIR)) { $n++; $npad = str_pad((string) $n, $padLength, '0', STR_PAD_LEFT); } $lockname = $npad.'-'.md5(uniqid(rand(), true)).'.'.$lockSuffix; if (@mkdir($lockname)) { $nlockdirs = glob($npad.'-*.'.$lockSuffix, GLOB_ONLYDIR) or array(); if (count($nlockdirs) != 1) { @rmdir($lockname); return false; } else { $prelockdirs = array(); $lockdirs = glob('*-*.'.$lockSuffix, GLOB_ONLYDIR) or array(); foreach($lockdirs as $lockdir) { list($m, ) = explode('-', $lockdir); $m = intval($m); if ($m < $n) $prelockdirs[] = $lockdir; } $t0 = time(); $t = 0; while (count($prelockdirs) != 0 and $t < $maxWait) { $prelockdirs = array(); $lockdirs = glob('*-*.'.$lockSuffix, GLOB_ONLYDIR) or array(); foreach($lockdirs as $lockdir) { list($m, ) = explode('-', $lockdir); $m = intval($m); if ($m < $n) $prelockdirs[] = $lockdir; } $t = time() - $t0; } if (count($prelockdirs) != 0) { @rmdir($lockname); return false; } else { return $lockname; } } } else { @rmdir($lockname); return false; } chdir($currentdir); } } ?> vlad
I was searching for the idea to get flock() to work without data loss, and unfortunately all i read here was not helpful to me, after some hard testing i still get some data lost or corrupted. So my suggestion is how to make a really data safe mechanism is to organize something like simple multithreaded queue daemon, that will listen for connections and wait for data to be sent and push it to some queue. Then process data from the queue and store it to file/sql or whatever we need. Another words: we get data queued and then processed and stored without any flock(); Also this gives us possibility to have centralized data sysytem (in case we need to collect data from different remote servers). Sure this will be not so easy as flock(), so people who count on proffessionalism and not luck, may find this helpful. Thnx. marc dot vanwoerkom
I ran into a loop because I just checked for true (= you got the lock) as return value of flock() and tried again when I got a false. function naive_wait_for_file($fp) { while (true) { if (flock($fp, LOCK_EX)) { return; } $k = rand(0, 20); usleep(round($k * 10000)); # k * 10ms } } Unfortunately in one case the $fp I put in was invalid, so I always got false and got stuck. Lesson: check if your $fp is valid before entering the loop, or look closer if you get a false. function wait_for_file($fp) { if ($fp === false) { return; } while (true) { if (flock($fp, LOCK_EX)) { return; } $k = rand(0, 20); usleep(round($k * 10000)); # k * 10ms } } joel at_sign purerave.com
I have found that if you open a currently locked file with 'w' or 'w+' ("file pointer at the beginning of the file and truncate the file to zero length") then it will not truncate the file when the lock is released and the file available. Example I used to test it: a.php: $fp = fopen( "/tmp/lock.txt", "w+" ); flock( $fp, LOCK_EX ); // exclusive lock $steps = 10; // write to the file for ($i=0; $i< $steps; $i++) { fwrite($fp, 'a '.time().' test '. $i."\n"); sleep(1); } flock( $fp, LOCK_UN ); // release the lock fclose( $fp ); ---------- b.php: $fp = fopen( "/tmp/lock.txt", "w+" ); flock( $fp, LOCK_EX ); // exclusive lock // ftruncate($fp, 0) is needed here! <---- $steps = 5; // write to the file for ($i=0; $i< $steps; $i++) { fwrite($fp, 'b '.time().' test '. $i."\n"); sleep(1); } flock( $fp, LOCK_UN ); // release the lock fclose( $fp ); Loading a.php then loading b.php right after will result in: b 1054075769 test 0 b 1054075770 test 1 b 1054075771 test 2 b 1054075772 test 3 b 1054075773 test 4 a 1054075764 test 5 a 1054075765 test 6 a 1054075766 test 7 a 1054075767 test 8 a 1054075768 test 9 As you can see, b.php does not truncate the file as the w+ would suggest if the file were instantly available. But only moves the pointer to the begining of the file. If b.php was loaded after a.php finished then there would be no "a ..." lines in the file, since it would be truncated. To fix this you have to add ftruncate($fp, 0) right after the flock. 'r+' and 'a' seem to work fine, though. maciej.trebacz
I have an idea about race condintions. How about make a seperate check for integrity of file somewhere in the middle of the script ? Let's take a look on this concept: 1. Some init code 2. Read whole using file() or other function 3. Do the rest that script want to do 4. After writing something to file, check for the file integrity. It can be done in several ways, depending on what you're putting to file. I'm doing file database's so it's rather easy. If the file fails this check, open it, lock it and use data read in (2) to rewrite it. Also a good idea will be putting this whole procedure into while...do, so you ensure that the file will be good. 5. Rest of the script It may also not be perfect, but I think it's a very big chance to prevent scripts to mess your files. ags one three seven
I am not certain how well this works (I've never had problems, but I haven't stress-tested it), but I have been using this solution for performing safe file locking and writing: if($fl = fopen($filepath)) if(flock($fl, LOCK_EX)) { fseek($fl, 0); ftruncate($fl, 0); fwrite($fl, $whateverdata); fflush($fl); flock($fl, LOCK_UN); fclose($fl); } This "cheats" and opens a file for writing by opening it for append (which doesn't truncate), then acquiring the lock first before performing any truncation or actual write operations. Thus, if the file cannot be successfully locked, it won't be changed at all. I usually like to assemble the data to be written in one large buffer first, so the file is open and locked for the shortest time possible. This is also friendlier to other scripts that are blocking on lock. lemonjuice
Hi, The discussions below address flock() in the context of managing integrity of file contents as well as the context of using flock() in combination with a dummy file to generally establish agreement on the access state of some other object. The following addresses the latter. I use this as a replacement for LOCK TABLES because during some transactional update statements I require the contents of other tables to freeze and transactions and tablelocks don't mix in mySQL / InnoDB. <?php class ReadWriteLock { const LOCK_PATH = "locks"; public static function Aquire($ID, $LockType = LOCK_SH, $WouldBlock = TRUE) { // Make sure the file exists and we have it opened. // We don't care about writing to the file. We just need a file reference that flock() can work on. // Also, on an OS level all thread's are sharing this file. We don't do access control in relation to this file. // So let's assume first that it already exists. $FileName = self::LOCK_PATH."/lock_$ID.lck"; if(($Resource = @fopen($FileName, "r")) === FALSE) // Ok, so this is the first time a thread acquires a lock to this $ID. Let's create the file. if(($Resource = @fopen($FileName, "w")) === FALSE) { // Ok, perhaps some thread created it between the two ifs. This class does not delete the file so it should now be there. if(($Resource = @fopen($FileName, "r")) === FALSE) return FALSE; } else { // #REF 1 // Ok, it exists now. Just for solidarity and prevention of whatever OS hickups we can possibly have // I want this thread to open it in r mode too. if(fclose($Resource) === FALSE) return FALSE; if(($Resource = @fopen($FileName, "r")) === FALSE) return FALSE; } // And this is really where the locking takes place. if(flock($Resource, $LockType, $WouldBlock)) return $Resource; fclose($Resource); return FALSE; } public static function Release($Resource) { if(fclose($Resource)) return TRUE; return FALSE; } } // Entering critical section if(($Lock = ReadWriteLock::Acquire("metadata")) === FALSE) die("Failed to either create or acquire the lock.") // Construct SQL statements from meta tables // Execute constructed SQL statements agains data tables // Leaving critical section if((ReadWriteLock::Release($Lock)) === FALSE) die("Failed to release the lock."); ?> A few notes about this: - As you can see, this just creates 'a' file to use as a reference for flock() to work on. My assumption here is that the operating system uses semaphores internally to implement Flock(). Of that, I am not sure however and I would appreciate any validation from an expert. - The problem of the existence of the lockfile is solved by simply not deleting them. Given that they are all 0-byte files in a specified folder and that they are only of a limited amount makes it something that works for my solution. Alternatively you could touch() the lockfile upon a succesful flock() and use a cron job to delete any files that have not been touched since, say, a day (or at least for the duration of the session timeout setting of your webserver). That would introduce a race condition for access on the actual file though which I prefer to exclude from the above. - flock() implements a low priority exclusive lock. This means that once the resource is locked in a shared mode, exclusive locks may be delayed indefinately if (and only if) a continuous abundance of shared lock requests come in so that every thread releases his shared lock after another thread has already gained shared access. For me, this is an ussue and I would appreciate any references to establish a high priority exclusive lock. Good luck, Juice ...Tastes like more! niels jaeckel
hi dranger at export dash japan dot com, you are right with your thoughts about process-switching. But I think the following kind of code would work in race-conditions, because it is very improbably that the CPU switches after just ONE statement: <?php // here my own lock-function WITHOUT flock() // you can call it a mutex... // lock ... compare to P(mutex) function lock($filename) { $filename .= '.lock'; while (true) { if (! file_exists($filename)) { if (! file_exists($filename)) { if (! file_exists($filename)) { return touch($filename); } } } } } // unlock ... compare to V(mutex) function unlock($filename) { unlink($filename . '.lock'); } ?> If you want to solve the reader/writer problem (many readers, just one exclusive writer) you can add some flock() - code instead of touch(...) hth administrator
Hello, I want to give an example how to lock file with two or more flags (for example reading and writing). IMPORTANT: each locking should be done separately, the correct way of using flock() is: <?php flock($fp, LOCK_EX); flock($fp, LOCK_SH); ?> and NOT like these: <?php flock($fp, LOCK_EX and LOCK_SH); flock($fp, LOCK_EX or LOCK_SH); flock($fp, LOCK_EX + LOCK_SH); ?> Furthermore if someone has not pay attention to function’s description - flock does not lock any file in the right way. The file is still accessible for reading/writing, in other words these functions: file(), file_get_contents() and even fopen($file, ‘r’) will ignore the lock. I think PHP mechanism works something like this: as soon as file lock was successful, the function flock() writes somewhere (in its own “DB†for example) that file handle is locked with some flag and nothing more. It is up to the developer to check if file is locked or not before doing any operations. Hope this post makes clear the flock() function’s working principles. Regards Vitali Simsive antti haapala
Further information on flock: The system is not restarted if a signal is delivered to the process, so flock will happily return false in case of SIGALRM, SIGFPE or something else.
m4v3r
Following is based on below comments. When something goes wrong, script will perform backup of writen data to randomly named file in temp dir. <? function safewrite($filename, $data){ $rand = microtime(); $rand = md5($rand); $temp = fopen("temp/$rand", "w"); fwrite($temp, $data); fclose($temp); $otw = fopen($filename, "a+"); if(flock($otw, LOCK_EX)){ ftruncate($otw, 0); if(!fwrite($otw, $data)) $err = 1; flock($otw, LOCK_UN); fclose($otw); } else { $err = 1; } if($err == 1 || (filesize($filename) == 0 && strlen($data) <> 0)){ die("<b>There was an error while writing to $filename. Contact site administrator!</b>"); } else { unlink("temp/$rand"); } } ?> Hope it helps. thanatos_101
flock() is disabled in some linux kernels (2.4 in my case), it always returns success. i'm using sysv semaphores to compensate. notice : knowing how to use icps and icprm are essential to debugging semaphore stuff!!! simple example for an exclusive only lock thingy // obtain exclusive lock function lock ($filename) { // get semaphore key $sem_key = ftok($filename, 'LOCK'); // get semaphore identifier $sem_id = sem_get($sem_key, 1); // acquire semaphore lock sem_acquire($sem_id); // return sem_id return $sem_id; } // release lock function unlock($sem_id) { // release semaphore lock sem_release($sem_id); } pentek_imre
flock isn't good in race conditions. I accept that it can correctly lock file and can correctly block php processes if file is locked, but anyway this function isn't the right way to manage a race condition. Let's have a look at this code: <?php $f=fopen($filename,"a+") or die(); flock($f,LOCK_EX) or die(); //here write some lines to the file -- not included //then close: flock($f,LOCK_UN) or die(); fclose($f) or die(); ?> Then generate a race situation with two php processes: 1: open file ok, no file found, create 2: open file ok, file found seek to the end (0lenght file so to the beginning) 1: lock file ok 2: flock waits since file is already locked. 1: write ok 1: unlock ok 2: flock ok this process now continues 1: fclose ok 2: write something, but due to prebuffering the file is now empty, so content written by 1 is now unconsidered, forgotten. 2: unlock ok 2: fclose ok file will have only the content from process 2 this is the situation if you use r+ too, no matter if you used fflush or not. conclusion: you may want to create a separate lock file! separate lock file will behave like in the previous example too, so let's try LOCK_NB and $wouldblock, file close and reopen. let's see this example: <?php $file="/tmp/phplockfile"; do { if(isset($f)) { flock($f,LOCK_UN); fclose($f); } $f=fopen($file,"a") or die(); flock($f,LOCK_EX+LOCK_NB,$W); // sleep(1); } while ($W==1); //lock is mine: echo $_SERVER["UNIQUE_ID"]." ".date("r")."\n"; sleep(1); echo $_SERVER["UNIQUE_ID"]." ".date("r")."\n"; //release the lock: flock($f,LOCK_UN); fclose($f); ?> I tried this code for 10 (ten) parellel processes. only three of them succeeds to lock the file and unlock it, the other seven quits with execuition timeout. uncommenting the sleep(1); won't help too, just execution will be longer (30 sec is counted not as real time but as cpu time) I tried random usleep too, as I remember this wasn't helped too. Remember that file close and reopen is a must becouse processes may write to the file, and this way these extra bytes will be considered too. 06-apr-2004 06:13
file locking with PHP over NFS shares simply does not work. Any attempt to use it will fail under any race condition. The only atomic file operation which is usable over NFS is mkdir() which will either create a folder or return a error status... So on NFS servers, forget flock(), use @mkdir() test the result, and if it fails, wait and retry. Shamely, PHP does not always have short sleeping periods and sleep(1) is too long for most PHP servers that limit execution time (expect to have complaints from users experimenting many 500 errors if the script is too long...) The bad thing about mkdir() is that it requires you to perform yourself a rmdir() when you're finished with the lock. If the rmdir() is not executed (because of execution time limitation), then the script will be aborted, leaving the locking directory present, and locking your site permanently. So you also need to check for the date status of the created folder, in a separate NFS operation, to see if the lock is not stale and to clean it (but when this maximum life of the lock has occured), race conditions will happen if several threads or PHP servers will want to drop the staled lock-directory. There's no simple solution, unless PHP releases a locking function based on mkdir() with AUTOMATIC rmdir() invokation when the PHP script engine terminates after a process or thread interrupt (unless it is killed -9, but a Web Server would normally not kill a script engine with a so brutal kill option). PHP is most often deployed by free hosting providers on parallel servers over NFS. Using MySQL to coordonate concurrent threads or process is overkill and lacks performance for some pages that have frequent accesses, notably above 500 requests per hour. PHP developers should really think about providing a working solution that will work on NFS, using mkdir() and rmdir() internally which is the only "magic" atomic operation allwoed there, and that is also performing the best (without requiring any complex client accesses to a SQL server...) j t atthedomain imen&org&uk
fackelkind honorsociety de wrote earlier a function to help with lock - but they should have used an incremental backoff solution - instead of sleeping for 30, w ould it not be nicer to sleep for 30 * n where n is the loop itteration?
christian dot wessels
dranger at export dash japan dot com is right, i once had the problem of a race condition in a database action because the script was very cpu-intensive so i looked around for some kind of mutex. I finally came up to this which uses no files (and no flock) and survived all of my tests (on linux): <?php function generate_shmkey ($string) { // Private: output unique integer based on $string return abs(crc32($string)); } function vmutex_lock ($lockId, $maxAcquire = 1) { // Public: take cover of semaphores and lock when possible global $shm_handle; $shm_key = generate_shmkey($lockId); if ($shm_handle = sem_get($shm_key, $maxAcquire)) { if (sem_acquire($shm_handle)) { return true; } } return false; } function vmutex_unlock ($lockId) { // Public: remove lock global $shm_handle; return sem_release($shm_handle); } ?> heshan
dranger at export dash japan dot com 's suggestion is correct and useful. My purpose is just preventing any other process from disturbing the current process which is rewriting a "cache" file, and flock() itself can help me to achieve this. $fp = fopen( "test.txt", "ab" ); if( $fp && flock( $fp, LOCK_EX ) ) { ftruncate( $fp, 0 ); fwrite( $fp, $part_one ); sleep( 10 ); // for test purpose, assume the whole writing process takes 10 seconds fwrite( $fp, $part_two ); fclose( $fp ); } it works well, I open 3 browser windows to request this page, and the third one actually takes more than 20( about 26 ) seconds to finish. and the content of cache.txt is correct and up-to-time. tested under: 1, windows, apache2, php4.4.1 2, freebsd, apache, php4.2.3 ivan
damasta at onwebworx dot net said that it is impossible to delete a lock file inside of a register_shutdown_function callback. I found a way to do that. You just have to use an absolute path to the file: <?php function shutdown_callback() { unlink(dirname($_SERVER['SCRIPT_FILENAME']) . "/lock.lock"); //otherwise it wouldn't work echo "<h1>Terminating</h1>\n"; } ?> Funny that echo works too, although PHP Manual says it shouldn't work :) chuck
Creating lock files and testing for their existence is absolutely wrong, NFS or not. The only file operations that are guaranteed to be atomic are mkdir() and symlink(). And those aren't atomic over NFS (or other network filesystems) either. Short answer: create lock directories, not files. Don't stat the directory to test, simply try to create it and see if it fails. And all locking bets are always off with NFS. arne
An addition to administrator at proxy-list dot org snippet below. To prevent unnecessary waiting check if the lock is obtained, if so skip the usleep(): $fp = fopen($logFileName, 'a'); $canWrite = false; //Waiting until file will be locked for writing while (!$canWrite) { $canWrite = flock($fp, LOCK_EX); // If lock not obtained sleep for 0 - 2000 miliseconds, to avoid colision if( !$canWrite ) { $miliSeconds = rand(0, 20); //1 u = 100 miliseconds usleep(round($miliSeconds*100000)); } } //file was locked so now we can store information fwrite($fp, $toSave); fclose($fp); kit dot lester
Amazing! these discussions are so similar to those back in the late '60s and early '70s about how to achieve file synchronization - especially across not-very-compatible platfoms. That was the big debate that led us to create databases, rollback and all, for exactly the same reasons. The only big difference between then and now is that these days we have the internet, whereas back then the discussion was in the letters columns of the computing journals. [E-mail only started to be available in the late 70s - and messy to use via gateways between different national nets.] PHP's flock() is a nice little hack for simple synchronization situations: trying to use it in big situations won't reliably work. If you need a bit of resilience in the application, you have to provide it yourself - which could be as simple as occasional manual backups of the files that could become corrupted. The mkdir/etc trick is a slightly bigger hack: but horridly dependant on atomicity, which is not guaranteed on guess-which-corporation's crummy operating systems. Consider switching to Unix/Linux/Mac. If you have a big data-handling problem needing high automatic resilience, then you need a database: sadly, no-one's come up with a better answer in 40 years of trying. If PHP+MySQL is too slow, you need either one of the faster expensive databases, or servelets. Or wait for CPUs to get fast enough for you. Or rethink the problem you're trying to solve [which is what I've done many many times over the decades: the resulting application is invariably better than anything I'd have achieved with the original poor problem-analysis.] ---------------- It's depressing how people will struggle with inventing a square wheel without checking whether it was invented before. "Those who know no history are condemned to repeat it." [Carlos Casteneda - but similar thoughts were expressed by Churchill, Napolean, Alexander of Macedon, and many others.] dranger
Also note that if you want to truncate a file, but make sure it's locked first, you DON'T need to use a separate lock file like the directions say. Use this instead: <?php $f=fopen("file", "r+"); flock($f, LOCK_EX) or die("Error! cant lock!"); ftruncate($f, 0); fwrite($f, $stuff); fclose($f); ?> But if you open a file with "w" or "w+" you WILL blow it away before you can lock it. bret
A poster above (below?) has the right idea with writing to a tempory file, then copying. However, achieving the 'right' to use the temp file in the first place is still a race condition issue: if (file_exists($temp_filename)) { $fd = fopen ($temp_filename, 'w'); } is still wrong, there's still a race condition between checking to whether the temp file exists and creating it. Using the 'x' option for opening is a way to avoid this. With the 'x' option chosen, the fopen will fail (returning FALSE) if the file already exists. $fd = fopen ($tmp, 'x'); if ($blocking == 1) { while ($fd === FALSE) { clearstatcache(); usleep(rand(5,70)); $fd = fopen ($tmp, 'x'); } } jeroenl
A possible workaround for the missing filelock mechanism. Thanks to Bret below who gave me the idea to use fopen() with the 'x' option. I use this for logging in gzipped logfiles up to many hundreds of KBs (and some small counter files) since a few weeks without any problems (yet). But again ... this is not 100% save and will fail with a possible locked file when the script is aborted while the lock is created. This can however - when needed - been solved by creating a lock cleanup e.g. based on filetime. And writing to the file is not 100% sure: we give up after trying for a while - but what is on the net anyhow :). If you need something else than adding to the end of a file, use a callback function for rewriting the filecontent. And you might want to change the loop count and sleep time. NB: tested (and using) on WinXP, Linux and FreeBSD. <? function fileWrite($file, &$str, $lockfile = null) { // ceate a lock file - if not possible someone else is active so wait (a while) $lock = ($lockfile)? $lockfile : $file.'.lock'; $lf = @fopen ($lock, 'x'); while ($lf === FALSE && $i++ < 20) { clearstatcache(); usleep(rand(5,85)); $lf = @fopen ($lf, 'x'); } // if lockfile (finally) has been created, file is ours, else give up ... if ($lf !== False) { $fp = fopen( $file, 'a'); fputs( $fp, $str); // or use a callback fclose( $fp); // and unlock fclose($lf); unlink($lock); } } ?> zmey
2pentek_imre at mailbox dot hu: flock() alone is not enough to handle race conditions, you will require the fseek() function too: <?php $f=fopen($filename,"a+") or die(); flock($f,LOCK_EX) or die(); // in case some other script has appended to the file while we were waiting on flock()... fseek($f,0,SEEK_END); //here write some lines to the file -- not included //then close: // flock($f,LOCK_UN) or die(); // fclose will remove all locks automatically fclose($f) or die(); ?> But the problem with interrupted script under apache still cannot be addressed gracefully. If the script is interrupted while writing to the file, the file's contents will get corrupt. Temporary files are nice, but they do not guarantee that your information will not be lost. For example: <?php // open file for read-write access $f=fopen($filename,"r+"); flock($f,LOCK_EX) or die(); $ft=fopen($filename.(getmypid()),"w") or die(); //here we copy data from the locked file to temporary one, modifying it in the process //then close temporary file: fclose($ft) or die(); // Now we have two files: temporary file and locked source file. // This trick works under *nix, but has some drawbacks. // For Windows, we must first unlink the original file. rename($filename.(getmypid()),$filename) or die(); // this will unlock the hidden, ex-original file under *nix, but will fail under win. fclose($f) or die(); ?> Under *nix, this code works just fine, but still can lead to data loss. Look: 1. We lock the original file. 2. Another copy of the script tries to lock the file (say, to modify some other part of file). It waits till we release the lock. 3. We make temporary file with modified data, and close it. 4. Rename succeeds, but in fact it works this way: 4.1 Original file's inode is marked as deleted, but the file is still on the disk because there are open handles to it. The file will be actually deleted only after the last handle closes. 4.2 Temporary file's inode is corrected so that its name is the name of the original file. What are the consequences? The second script, that was patiently waiting for us to unlock the file, gets access to the file, BUT: the file is not the new, updated version! The script's handle still points to the original file, although it is not accessible by any other program. So the script happily processes the OLD version of the original file, and replaces the files once again. Voila - the updates made by the first script are all lost! Under Windows, we will not be able to rename the temporary file without unlinking the original file first. Besides, we will not be able to unlink the file without unlocking it (Windows uses mandatory locking)! So we will not be able to reliably replace the original file with the temporary one without some trickery (e.g., use of external lock files). mic
2 rehfeld.us in your code <?php ... $timeStart = microtime_float(); do { if ((microtime_float() - $timeStart) > $timeLimit) break; $locked = @mkdir($lockDir); } while ($locked === false); ... ?> $timeStart and return value of microtime_float() is _float_, i.e. for example $timestart = 1101991367.393253 and microtime_float() - $timeStart will be like 0.00123 but $timeLimit is _integer_ and much bigger So code will work but execution timeout ocured. you must set value of $timeLimit in float, i.e. $timeLimit = 0.03 rehfeld.us
<?php /* * I hope this is usefull. * If mkdir() is atomic, * then we do not need to worry about race conditions while trying to make the lockDir, * unless of course were writing to NFS, for which this function will be useless. * so thats why i pulled out the usleep(rand()) peice from the last version * * Again, its important to tailor some of the parameters to ones indivdual usage * I set the default $timeLimit to 3/10th's of a second (maximum time allowed to achieve a lock), * but if your writing some extrememly large files, and/or your server is very slow, you may need to increase it. * Obviously, the $staleAge of the lock directory will be important to consider as well if the writing operations might take a while. * My defaults are extrememly general and you're encouraged to set your own * * $timeLimit is in microseconds * $staleAge is in seconds * * */ function microtime_float() { list($usec, $sec) = explode(' ', microtime()); return ((float)$usec + (float)$sec); } function locked_filewrite($filename, $data, $timeLimit = 300000, $staleAge = 5) { ignore_user_abort(1); $lockDir = $filename . '.lock'; if (is_dir($lockDir)) { if ((time() - filemtime($lockDir)) > $staleAge) { rmdir($lockDir); } } $locked = @mkdir($lockDir); if ($locked === false) { $timeStart = microtime_float(); do { if ((microtime_float() - $timeStart) > $timeLimit) break; $locked = @mkdir($lockDir); } while ($locked === false); } $success = false; if ($locked === true) { $fp = @fopen($filename, 'a'); if (@fwrite($fp, $data)) $success = true; @fclose($fp); rmdir($lockDir); } ignore_user_abort(0); return $success; } ?> |
Change Language
basename chgrp chmod chown clearstatcache copy delete dirname disk_free_space disk_total_space diskfreespace fclose feof fflush fgetc fgetcsv fgets fgetss file_exists file_get_contents file_put_contents file fileatime filectime filegroup fileinode filemtime fileowner fileperms filesize filetype flock fnmatch fopen fpassthru fputcsv fputs fread fscanf fseek fstat ftell ftruncate fwrite glob is_dir is_executable is_file is_link is_readable is_uploaded_file is_writable is_writeable lchgrp lchown link linkinfo lstat mkdir move_uploaded_file parse_ini_file pathinfo pclose popen readfile readlink realpath rename rewind rmdir set_file_buffer stat symlink tempnam tmpfile touch umask unlink |
