mirror of
https://git.mirrors.martin98.com/https://github.com/actions/toolkit
synced 2026-04-05 03:43:14 +08:00
Support upload from named pipes (#748)
Named pipes report file size as 0, which leads to reading the whole content into memory (0 is less than 64K). This adds additional check to make sure that the passed in path is not a named pipe, and in that case opts for the create-temp-file-to-gzip code path. When running on GitHub Actions infrastructure on `windows` node, named pipes can be created using `mkfifo` from MSYS2. In that case `fs.Stats`s `isFIFO()` returns `false`, and not `true` as expected. This case is detected by `process.platform` being `win32` and the passed file having length of 0. As a side note, when MSYS2's `mkfifo` is run, a pipe file is created: ``` prw-rw-rw- 1 User None 0 Mar 31 12:58 pipe ``` If `fs.stat` is invoked at this point `ENOENT` error will be thrown. As soon as the pipe is written to, this pipe file is replaced by two same- named files: ``` -rw-r--r-- 1 User None 0 Mar 31 13:00 pipe -rw-r--r-- 1 User None 0 Mar 31 13:00 pipe ``` And at this point `fs.stat` `isFIFO()` returns `false`. Even though the file acts as a named pipe.
This commit is contained in:
@@ -219,16 +219,22 @@ export class UploadHttpClient {
|
||||
httpClientIndex: number,
|
||||
parameters: UploadFileParameters
|
||||
): Promise<UploadFileResult> {
|
||||
const totalFileSize: number = (await stat(parameters.file)).size
|
||||
const fileStat: fs.Stats = await stat(parameters.file)
|
||||
const totalFileSize = fileStat.size
|
||||
// on Windows with mkfifo from MSYS2 stats.isFIFO returns false, so we check if running on Windows node and
|
||||
// if the file has size of 0 to compensate
|
||||
const isFIFO =
|
||||
fileStat.isFIFO() || (process.platform === 'win32' && totalFileSize === 0)
|
||||
let offset = 0
|
||||
let isUploadSuccessful = true
|
||||
let failedChunkSizes = 0
|
||||
let uploadFileSize = 0
|
||||
let isGzip = true
|
||||
|
||||
// the file that is being uploaded is less than 64k in size, to increase throughput and to minimize disk I/O
|
||||
// the file that is being uploaded is less than 64k in size to increase throughput and to minimize disk I/O
|
||||
// for creating a new GZip file, an in-memory buffer is used for compression
|
||||
if (totalFileSize < 65536) {
|
||||
// with named pipes the file size is reported as zero in that case don't read the file in memory
|
||||
if (!isFIFO && totalFileSize < 65536) {
|
||||
const buffer = await createGZipFileInBuffer(parameters.file)
|
||||
|
||||
//An open stream is needed in the event of a failure and we need to retry. If a NodeJS.ReadableStream is directly passed in,
|
||||
@@ -287,7 +293,8 @@ export class UploadHttpClient {
|
||||
let uploadFilePath = tempFile.path
|
||||
|
||||
// compression did not help with size reduction, use the original file for upload and delete the temp GZip file
|
||||
if (totalFileSize < uploadFileSize) {
|
||||
// for named pipes totalFileSize is zero, this assumes compression did help
|
||||
if (!isFIFO && totalFileSize < uploadFileSize) {
|
||||
uploadFileSize = totalFileSize
|
||||
uploadFilePath = parameters.file
|
||||
isGzip = false
|
||||
|
||||
Reference in New Issue
Block a user