Build a PHP FTP Synchronizer — Step‑by‑Step GuideKeeping files synchronized between local machines, staging servers, and production environments is a common task for developers and sysadmins. A PHP FTP synchronizer can automate uploads, downloads, and two‑way syncs using standard FTP or FTPS — useful when you can’t use SSH/SFTP or when PHP is the available runtime on the host. This guide walks through planning, architecture, implementation, and deployment of a reliable PHP FTP synchronizer.
What this guide covers
- Requirements and design decisions
- Secure connection options (FTP vs FTPS)
- Directory scanning and change detection strategies
- Transfer logic and conflict resolution
- Error handling, retries, and logging
- Performance considerations and optimizations
- Example implementation with well‑commented PHP code
- Testing and deployment tips
1. Requirements & design decisions
Before coding, clarify what the synchronizer must do. Answer these questions:
- Direction: one‑way (local → remote) or two‑way (bidirectional)?
- Scope: single directory, multiple directories, or entire trees?
- File matching: include/exclude patterns (by extension, name, size, mtime)?
- Conflict rules: remote wins, local wins, or keep both (e.g., rename)?
- Transport: plain FTP or secure FTPS/TLS? (Prefer FTPS where possible.)
- Scheduling: run on demand, cron job, or daemons with watchers?
- Resource limits: concurrent transfers, memory/CPU caps, resume support?
- Logging/auditing: how detailed should logs be and where stored?
Decide early whether to build a simple script for ad‑hoc tasks or a robust tool with resumable transfers, queues, and persistent state (e.g., SQLite).
2. Secure connection options
FTP sends credentials and file contents in clear text unless wrapped by TLS (FTPS). When possible, use FTPS (FTP over TLS) or switch to SFTP (SSH File Transfer Protocol) — though SFTP requires an SSH client library (e.g., phpseclib) rather than the standard FTP extension.
- Use FTPS (FTP with TLS) via PHP’s ftp_ssl_connect when the server supports it.
- Fallback: plain ftp_connect only if FTPS is unavailable and network is trusted.
- Alternative: SFTP with phpseclib if server allows SSH and you need stronger security.
Also avoid embedding credentials in code; use environment variables or config files with restricted permissions.
3. Directory scanning and change detection
A synchronizer needs to detect what changed since the last run. Methods:
- Timestamp comparison (mtime): quick, but clock skew between hosts can cause issues.
- Size plus mtime: more robust against trivial timestamp changes.
- Checksums (MD5/SHA): most reliable but more CPU and network intensive.
- Local state database: store a local snapshot (SQLite/JSON) of file metadata (path, size, mtime, checksum) and compare on each run.
Recommendation: combine mtime+size with a periodic checksum verification for critical files.
Example metadata schema for SQLite:
- path TEXT PRIMARY KEY
- size INTEGER
- mtime INTEGER
- checksum TEXT
- last_synced INTEGER
4. Transfer logic & conflict resolution
Basic steps for one run (one‑way local→remote):
- Scan local directory recursively and build list of candidates (apply include/exclude patterns).
- Connect to remote and optionally build remote file list (or request file metadata as needed).
- Compare local vs remote using chosen strategy (mtime/size/checksum/state DB).
- For files that need upload: create remote directories as needed, then upload.
- For remote files not present locally: delete (if desired) or skip based on policy.
- Update state DB and logs.
For two‑way sync:
- Determine which side has the newest version per file.
- Apply conflict policy:
- Local wins: always overwrite remote with local copy.
- Remote wins: always overwrite local with remote copy.
- Rename: keep both by renaming the older (e.g., file.txt → file.conflict.20250901.txt).
- Manual review: queue conflicting files for human inspection.
Atomicity: upload to a temp filename then rename on success to avoid partially written files being served.
5. Error handling, retries, and logging
Robustness requires clear error handling:
- Wrap FTP operations with retries and exponential backoff.
- Handle partial uploads: detect size mismatch and resume if server supports REST (FTP REST command).
- Timeouts: set reasonable connection and transfer timeouts.
- Logging: record actions (uploads/downloads/deletes), results, timestamps, and error messages. Use rotating logs or limit size.
- Monitoring: send alerts or exit codes when runs fail persistently.
Log example fields:
- timestamp, action, path, direction, status, bytes, duration, error_message
6. Performance considerations
- Parallel transfers: use multiple FTP connections for concurrent uploads/downloads. Beware of server connection limits.
- Batch directory listings: minimize repeated LIST or MLSD commands. MLSD is preferred for machine‑readable listings.
- Use passive mode (PASV) for NAT friendliness.
- Throttling: add bandwidth limits if required to avoid saturating the network.
- Resume support: implement REST for resuming large file transfers.
- Caching: cache remote listings and metadata between runs for speed.
7. Implementation — example PHP synchronizer
Below is a concise but complete PHP example implementing a one‑way (local → remote) synchronizer using FTPS when available, with SQLite state tracking, retries, and logging. It uses the PHP FTP extension and PDO for SQLite.
Note: adjust configuration variables and ensure php_ftp and pdo_sqlite extensions are enabled on your PHP build.
<?php // config $cfg = [ 'host' => getenv('FTP_HOST') ?: 'ftp.example.com', 'port' => 21, 'username' => getenv('FTP_USER') ?: 'user', 'password' => getenv('FTP_PASS') ?: 'pass', 'use_ssl' => true, // try FTPS first 'remote_root' => '/public_html', 'local_root' => __DIR__ . '/sync_folder', 'log_file' => __DIR__ . '/sync.log', 'db_file' => __DIR__ . '/sync_state.sqlite', 'include_patterns' => ['~.(php|html|css|js|png|jpg|gif)$~i'], 'exclude_patterns' => ['~^/node_modules/~'], 'max_retries' => 3, 'retry_delay' => 1, // seconds (exponential backoff) ]; // simple logger function logMsg($cfg, $level, $msg) { $line = sprintf("[%s] %-5s %s ", date('c'), strtoupper($level), $msg); file_put_contents($cfg['log_file'], $line, FILE_APPEND | LOCK_EX); } // open or create state DB $pdo = new PDO('sqlite:' . $cfg['db_file']); $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $pdo->exec("CREATE TABLE IF NOT EXISTS files ( path TEXT PRIMARY KEY, size INTEGER, mtime INTEGER, checksum TEXT, last_synced INTEGER )"); // connect FTP (try FTPS then FTP) function ftpConnect($cfg) { if ($cfg['use_ssl'] && function_exists('ftp_ssl_connect')) { $conn = @ftp_ssl_connect($cfg['host'], $cfg['port'], 30); if ($conn) return $conn; } $conn = @ftp_connect($cfg['host'], $cfg['port'], 30); return $conn; } $ftp = ftpConnect($cfg); if (!$ftp) { logMsg($cfg, 'error', 'Could not connect to FTP server'); exit(1); } if (!@ftp_login($ftp, $cfg['username'], $cfg['password'])) { logMsg($cfg, 'error', 'FTP login failed'); ftp_close($ftp); exit(1); } ftp_pasv($ftp, true); // helper: walk local dir function walkLocal($root, $include, $exclude) { $files = []; $it = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($root, FilesystemIterator::SKIP_DOTS)); foreach ($it as $f) { $rel = '/' . ltrim(str_replace($root, '', $f->getPathname()), '/\'); $skip = false; foreach ($exclude as $pat) if (preg_match($pat, $rel)) { $skip = true; break; } if ($skip) continue; $ok = empty($include); foreach ($include as $pat) if (preg_match($pat, $rel)) { $ok = true; break; } if (!$ok) continue; $files[$rel] = ['size' => $f->getSize(), 'mtime' => $f->getMTime(), 'full' => $f->getPathname()]; } return $files; } // create remote dir recursively function ensureRemoteDir($ftp, $dir) { $parts = explode('/', trim($dir, '/')); $cwd = ''; foreach ($parts as $p) { $cwd .= '/' . $p; if (@ftp_chdir($ftp, $cwd) === false) { ftp_mkdir($ftp, $cwd); } } } // upload with retries and atomic rename function uploadFile($ftp, $localFile, $remotePath, $cfg) { $tmp = $remotePath . '.tmp'; $tries = 0; while ($tries < $cfg['max_retries']) { $tries++; $stream = fopen($localFile, 'r'); if ($stream === false) return [false, "Cannot open local file"]; $ok = ftp_fput($ftp, $tmp, $stream, FTP_BINARY); fclose($stream); if ($ok) { // rename if (@ftp_rename($ftp, $tmp, $remotePath)) { return [true, null]; } else { return [false, "Uploaded but rename failed"]; } } sleep($cfg['retry_delay'] * $tries); } return [false, "Upload failed after retries"]; } // main sync $localFiles = walkLocal($cfg['local_root'], $cfg['include_patterns'], $cfg['exclude_patterns']); foreach ($localFiles as $rel => $meta) { $remotePath = rtrim($cfg['remote_root'], '/') . $rel; // check state DB $stmt = $pdo->prepare('SELECT size, mtime, checksum FROM files WHERE path = :p'); $stmt->execute([':p' => $rel]); $row = $stmt->fetch(PDO::FETCH_ASSOC); $needsUpload = true; if ($row) { if ($row['size'] == $meta['size'] && $row['mtime'] == $meta['mtime']) { $needsUpload = false; // unchanged } } if ($needsUpload) { // ensure remote dir $rdir = dirname($remotePath); ensureRemoteDir($ftp, $rdir); list($ok, $err) = uploadFile($ftp, $meta['full'], $remotePath, $cfg); if ($ok) { $checksum = hash_file('sha256', $meta['full']); $stmt = $pdo->prepare('REPLACE INTO files (path,size,mtime,checksum,last_synced) VALUES (:p,:s,:m,:c,:t)'); $stmt->execute([ ':p' => $rel, ':s' => $meta['size'], ':m' => $meta['mtime'], ':c' => $checksum, ':t' => time() ]); logMsg($cfg, 'info', "Uploaded {$rel} ({$meta['size']} bytes)"); } else { logMsg($cfg, 'error', "Failed upload {$rel}: {$err}"); } } else { logMsg($cfg, 'debug', "Skipping unchanged {$rel}"); } } ftp_close($ftp); logMsg($cfg, 'info', 'Sync finished'); 8. Testing and validation
- Unit test helper functions (path normalization, include/exclude matching).
- Integration test with a test FTP server (vsftpd, ProFTPD, or a containerized server) to validate passive/active modes and MLSD support.
- Test interrupted transfers and resume behavior.
- Test conflict scenarios when doing two‑way syncs.
If you can, run the sync in a dry‑run mode first (log planned actions without executing) to confirm rules.
9. Deployment & scheduling
- Use cron to schedule runs (e.g., every 5 minutes for frequent syncs).
- For near‑real‑time sync, use inotify (Linux) to detect local changes and trigger the script; still batch uploads to avoid too many small transfers.
- Run with a dedicated user and limit filesystem permissions for security.
- Rotate logs and back up the SQLite state.
Example cron entry (every 10 minutes): */10 * * * * /usr/bin/php /path/to/ftp_sync.php >> /var/log/ftp_sync_cron.log 2>&1
10. Further enhancements
- Add SFTP support via phpseclib for stronger security.
- Implement file compression for transfer of many small files (create tar.gz and transfer single archive, then extract remotely if you have shell access).
- Add a web UI for monitoring and resolving conflicts.
- Use a queue (RabbitMQ/Redis) for large scale parallel transfers.
- Integrate with CI/CD pipelines for deployment automation.
This guide showed how to design, implement, test, and deploy a PHP FTP synchronizer. The included example is a practical starting point — adapt include/exclude patterns, conflict rules, and security settings to match your environment.
Leave a Reply