File Manipulation in Node.js

Introduction

File manipulation is one of the most important tasks in backend development. Whether you are building a web server, command-line tool, or automated system, you will often need to manage files — renaming, deleting, copying, or moving them. Node.js makes these operations simple and efficient through its built-in File System (fs) module.

In Node.js, file manipulation is asynchronous by default. This means operations like deleting, copying, or renaming files do not block the main thread, allowing your application to remain responsive. The asynchronous nature of Node.js file handling makes it ideal for applications that deal with large files, frequent uploads, or background data processing.

This post will explore the core concepts and practical uses of file manipulation in Node.js, covering everything from simple file operations to advanced error handling, asynchronous best practices, and real-world use cases.

---

Understanding the fs Module

Before performing file manipulations, you need to understand the fs (file system) module. It provides a powerful set of functions to interact with the operating system’s file system, enabling developers to read, write, update, delete, and modify files and directories.

You can import the module as follows:

const fs = require('fs');

For modern promise-based operations, Node.js also offers a Promises API under fs.promises:

const fs = require('fs').promises;

The promise-based version is particularly useful for asynchronous code that uses async/await syntax, making file manipulation code cleaner and easier to maintain.

---

Renaming Files with fs.rename()

Renaming a file is a common operation when managing uploads, backups, or log rotation. Node.js provides the fs.rename() method to rename files asynchronously.

Example Using Callbacks

const fs = require('fs');

fs.rename('oldname.txt', 'newname.txt', (err) => {
  if (err) {
console.error('Error renaming file:', err);
return;
  }
  console.log('File renamed successfully.');
});

This function takes three parameters:

1. The current name (path) of the file.
2. The new name (path).
3. A callback that handles success or error.

If the target file name already exists, the existing file will be overwritten by default.

Example Using Promises

const fs = require('fs').promises;

async function renameFile() {
  try {
await fs.rename('oldname.txt', 'newname.txt');
console.log('File renamed successfully.');
  } catch (err) {
console.error('Error renaming file:', err);
  }
}

renameFile();

Common Use Cases for Renaming

• Renaming uploaded files based on user ID or timestamp.
• Rotating log files periodically.
• Changing file extensions after processing data.

---

Deleting Files with fs.unlink()

Deleting unnecessary files helps manage disk space and maintain a clean file system. Node.js provides fs.unlink() to delete files asynchronously.

Example Using Callbacks

const fs = require('fs');

fs.unlink('temp.txt', (err) => {
  if (err) {
console.error('Error deleting file:', err);
return;
  }
  console.log('File deleted successfully.');
});

Example Using Promises and Async/Await

const fs = require('fs').promises;

async function deleteFile() {
  try {
await fs.unlink('temp.txt');
console.log('File deleted successfully.');
  } catch (err) {
if (err.code === 'ENOENT') {
  console.error('File not found.');
} else {
  console.error('Error deleting file:', err);
}
  }
}

deleteFile();

In this example, the ENOENT error code means the file doesn’t exist. Handling such cases gracefully ensures your application doesn’t crash when attempting to delete missing files.

Common Use Cases for Deleting Files

• Cleaning up temporary uploads.
• Removing outdated log files.
• Deleting user-generated content upon account deletion.
• Managing cache or backup folders.

---

Copying Files with fs.copyFile()

Copying files is another frequent task in backend systems. Node.js provides fs.copyFile() for asynchronous file copying.

Example Using Callbacks

const fs = require('fs');

fs.copyFile('source.txt', 'destination.txt', (err) => {
  if (err) {
console.error('Error copying file:', err);
return;
  }
  console.log('File copied successfully.');
});

Example Using Promises and Async/Await

const fs = require('fs').promises;

async function copyFile() {
  try {
await fs.copyFile('source.txt', 'destination.txt');
console.log('File copied successfully.');
  } catch (err) {
console.error('Error copying file:', err);
  }
}

copyFile();

Using Flags for Copy Behavior

You can control how copying behaves using flags. For example:

const fs = require('fs');
const constants = require('fs').constants;

fs.copyFile('source.txt', 'destination.txt', constants.COPYFILE_EXCL, (err) => {
  if (err) {
console.error('Error:', err);
return;
  }
  console.log('File copied only if destination does not exist.');
});

The COPYFILE_EXCL flag prevents overwriting the destination file if it already exists. This is especially useful for backup operations.

---

Moving Files in Node.js

Node.js doesn’t have a built-in “move” method, but you can simulate moving by renaming a file to a new directory path.

Example

const fs = require('fs').promises;

async function moveFile() {
  try {
await fs.rename('uploads/image.png', 'archive/image.png');
console.log('File moved successfully.');
  } catch (err) {
console.error('Error moving file:', err);
  }
}

moveFile();

If the source and destination are on different disks or volumes, you’ll need to copy the file and then delete the original:

async function moveFileCrossDisk(src, dest) {
  try {
await fs.copyFile(src, dest);
await fs.unlink(src);
console.log('File moved successfully across disks.');
  } catch (err) {
console.error('Error moving file:', err);
  }
}

---

Checking if a File Exists

Before renaming, deleting, or copying a file, it’s often important to check whether the file exists. The recommended way to do this is using fs.access().

Example Using Promises

const fs = require('fs').promises;

async function fileExists(path) {
  try {
await fs.access(path);
console.log('File exists.');
return true;
  } catch {
console.log('File does not exist.');
return false;
  }
}

fileExists('example.txt');

fs.access() is non-blocking and efficient, and it can also check for specific permissions like read or write access.

---

Reading and Modifying File Metadata

Every file has metadata, including size, creation time, and modification date. You can access this information using fs.stat().

Example

const fs = require('fs').promises;

async function getFileStats() {
  try {
const stats = await fs.stat('example.txt');
console.log('File size:', stats.size);
console.log('Created at:', stats.birthtime);
console.log('Last modified:', stats.mtime);
  } catch (err) {
console.error('Error getting stats:', err);
  }
}

getFileStats();

This data can help you make decisions about whether to archive, delete, or process a file.

---

Working with Directories

File manipulation often involves managing directories in addition to individual files. The fs module allows you to create, read, and remove directories.

Creating a Directory

const fs = require('fs').promises;

async function createDirectory() {
  try {
await fs.mkdir('uploads', { recursive: true });
console.log('Directory created successfully.');
  } catch (err) {
console.error('Error creating directory:', err);
  }
}

createDirectory();

The { recursive: true } option ensures that intermediate directories are created if they don’t exist.

Deleting a Directory

const fs = require('fs').promises;

async function deleteDirectory() {
  try {
await fs.rmdir('uploads');
console.log('Directory deleted successfully.');
  } catch (err) {
console.error('Error deleting directory:', err);
  }
}

In Node.js 14 and above, you can also use fs.rm() with { recursive: true } to delete non-empty directories.

---

File Manipulation with Async/Await

When dealing with multiple file operations, async/await provides a clean, readable way to chain asynchronous tasks.

Example: Copy, Rename, and Delete Sequence

const fs = require('fs').promises;

async function manipulateFiles() {
  try {
await fs.copyFile('original.txt', 'backup.txt');
console.log('Backup created.');
await fs.rename('original.txt', 'renamed.txt');
console.log('File renamed.');
await fs.unlink('backup.txt');
console.log('Backup deleted.');
  } catch (err) {
console.error('File operation failed:', err);
  }
}

manipulateFiles();

Each operation waits for the previous one to complete, ensuring predictable results.

---

Error Handling in File Manipulation

File manipulation can fail for many reasons: missing files, permission issues, or disk errors. Always handle errors gracefully.

Common Error Codes

• ENOENT – File or directory not found.
• EACCES – Permission denied.
• EEXIST – File already exists.
• EBUSY – Resource busy or locked.

Example of handling different errors:

async function safeDelete(path) {
  try {
await fs.unlink(path);
console.log('File deleted.');
  } catch (err) {
switch (err.code) {
  case 'ENOENT':
    console.log('File does not exist.');
    break;
  case 'EACCES':
    console.log('Permission denied.');
    break;
  default:
    console.error('Unexpected error:', err);
}
  }
}

safeDelete('temp.txt');

Good error handling prevents crashes and helps maintain data integrity.

---

Working with File Streams for Large Files

When copying or moving large files, it’s better to use streams instead of reading the whole file into memory.

Example of Copying with Streams

const fs = require('fs');

function copyLargeFile(source, destination) {
  const readStream = fs.createReadStream(source);
  const writeStream = fs.createWriteStream(destination);

  readStream.pipe(writeStream);

  writeStream.on('finish', () => {
console.log('Large file copied successfully.');
  });

  readStream.on('error', err => console.error('Read error:', err));
  writeStream.on('error', err => console.error('Write error:', err));
}

copyLargeFile('largevideo.mp4', 'backup/largevideo.mp4');

Streams process data in chunks, which reduces memory usage and improves performance for large files.

---

Monitoring File Changes

Sometimes you may want to monitor file changes automatically and perform actions like moving, renaming, or deleting.

Using fs.watch()

const fs = require('fs');

fs.watch('logs', (eventType, filename) => {
  console.log(File changed: ${filename} (${eventType}));
});

This is useful for:

• Monitoring uploads.
• Watching log files for updates.
• Triggering automatic backups or deletions.

---

Real-World File Manipulation Scenarios

1. Managing Uploaded Files

When users upload files, you often need to rename them with unique identifiers to avoid conflicts.

const fs = require('fs').promises;

async function handleUpload(tempPath, userId) {
  const newName = upload_${userId}_${Date.now()}.png;
  await fs.rename(tempPath, uploads/${newName});
  console.log('File uploaded and renamed:', newName);
}

2. Log Rotation

Applications that generate logs may need to rename or move logs daily.

async function rotateLogs() {
  const oldPath = 'logs/app.log';
  const newPath = logs/app_${Date.now()}.log;
  await fs.rename(oldPath, newPath);
  console.log('Log rotated.');
}

3. Cleaning Up Temporary Files

Automated cleanup ensures unused files do not consume disk space.

async function cleanTemp() {
  const tempFiles = ['temp1.txt', 'temp2.txt', 'temp3.txt'];
  for (const file of tempFiles) {
try {
  await fs.unlink(file);
  console.log(Deleted: ${file});
} catch (err) {
  if (err.code !== 'ENOENT') console.error('Error deleting:', file);
}
  }
}

---

Performance Tips

1. Use asynchronous methods to prevent blocking the event loop.
2. Use streams for large files to manage memory efficiently.
3. Batch file operations with Promise.all for concurrent processing.
4. Avoid synchronous file operations on the server.
5. Gracefully handle errors to ensure reliability.
6. Check file existence before performing destructive operations.
7. Use path.join() for cross-platform file paths.
8. Limit concurrent file operations to prevent disk overload.
9. Log every file operation for debugging and audit purposes.
10. Cache metadata when accessing the same files frequently.

---

File Manipulation Security Considerations

• Validate file paths: Prevent directory traversal attacks (e.g., ../../etc/passwd).
• Limit access: Use proper permissions for file read/write operations.
• Sanitize filenames: Remove unsafe characters from user-provided filenames.
• Avoid overwriting important files: Use flags like COPYFILE_EXCL.
• Handle sensitive data carefully: Securely delete files containing private information.

---

Testing File Manipulation Code

When testing file operations, use mock libraries or temporary directories to prevent modifying real data.

Example using Jest:

const fs = require('fs').promises;

test('should rename a file', async () => {
  await fs.writeFile('test.txt', 'hello');
  await fs.rename('test.txt', 'renamed.txt');
  const data = await fs.readFile('renamed.txt', 'utf8');
  expect(data).toBe('hello');
  await fs.unlink('renamed.txt');
});

Testing ensures that your file manipulation logic works across environments safely.

---

Best Practices for File Manipulation

1. Use Async/Await for readability and non-blocking behavior.
2. Always handle errors with try…catch.
3. Use Promises API (fs.promises) for modern and cleaner syntax.
4. Check for file existence before deleting or renaming.
5. Avoid overwriting important files.
6. Use streams for large file transfers.
7. Log all file operations.
8. Implement file cleanup routines.
9. Handle cross-platform paths using path.join().
10. Ensure proper security when handling user uploads.