,我想出来了,
这是丢失的那块。一旦我清理了我的代码,我将发布一个答案,希望下一个必须处理这个问题的可怜人不必经历我经历过的同样的地狱;)
$command = $client->getCommand('UploadPart', array(
'Bucket' => 'the-bucket-name',
'Key' => $key,
'PartNumber' => $partNumber,
'UploadId' => $uploadId,
'Body' => '',
));
$signedUrl = $client->createPresignedRequest($command, '+20 minutes');
$presignedUrl = (string)$signedUrl->getUri();
return response()->json(['url' => $presignedUrl]);
我正在尝试通过使用S3选项来配置我的服务器,以便使用Uppy将多部分上传到AWS S3。https://uppy.io/docs/aws-s3-multipart/#createMultipartUpload-file。
这就是我想走这条路线的地方,https://github.com/transloadit/uppy/issues/1189#issuecomment-445521442。
我不明白这一点,我觉得其他人也被困在没有答案的情况下,所以我发布了到目前为止我在试图让Uppy使用Laravel/Vue进行多部分上传时所想出的内容。
对于Vue组件,我有以下内容:
<template>
<div>
<a id="uppy-trigger" @click="isUppyOpen = !isUppyOpen">Open Uppy</a>
<dashboard-modal
:uppy="uppy"
:open="isUppyOpen"
:props="{trigger: '#uppy-trigger'}"
/>
</div>
</template>
<script>
import Uppy from '@uppy/core'
import AwsS3Multipart from '@uppy/aws-s3-multipart';
import '@uppy/core/dist/style.css';
import '@uppy/dashboard/dist/style.css';
export default {
components: {
'dashboard-modal': DashboardModal,
},
data() {
return {
isUppyOpen: false,
}
},
computed: {
// Uppy Instance
uppy: () => new Uppy({
logger: Uppy.debugLogger
}).use(AwsS3Multipart, {
limit: 4,
companionUrl: 'https://mysite.local/',
}),
},
beforeDestroy () {
this.uppy.close();
},
}
</script>
然后,对于路由,我将其添加到我的web.php文件中。
// AWS S3 Multipart Upload Routes
Route::name('s3.multipart.')->prefix('s3/multipart')
->group(function () {
Route::post('/', ['as' => 'createMultipartUpload', 'uses' => 'AwsS3MultipartController@createMultipartUpload']);
Route::get('{uploadId}', ['as' => 'getUploadedParts', 'uses' => 'AwsS3MultipartController@getUploadedParts']);
Route::get('{uploadId}/{partNumber}', ['as' => 'signPartUpload', 'uses' => 'AwsS3MultipartController@signPartUpload']);
Route::post('{uploadId}/complete', ['as' => 'completeMultipartUpload', 'uses' => 'AwsS3MultipartController@completeMultipartUpload']);
Route::delete('{uploadId}', ['as' => 'abortMultipartUpload', 'uses' => 'AwsS3MultipartController@abortMultipartUpload']);
});
基本上,我已经将"companionUrl“设置为"https://mysite.local/"”,当将多部分上传文件上传到这些路由时,Uppy将发送五个请求,即"https://mysite.local/s3/multipart/createMultipartUpload"“。
然后,我创建了一个控制器来处理请求:
<?php
namespace App\Http\Controllers;
use Aws\S3\S3Client;
use Illuminate\Http\Request;
class AwsS3MultipartController extends Controller
{
public function createMultipartUpload(Request $request)
{
$client = new S3Client([
'version' => 'latest',
'region' => 'us-east-1',
]);
$key = $request->has('filename') ? $request->get('filename') : null;
$type = $request->has('type') ? $request->get('type') : null;
if (!is_string($key)) {
return response()->json(['error' => 's3: filename returned from "getKey" must be a string'], 500);
}
if (!is_string($type)) {
return response()->json(['error' => 's3: content type must be a string'], 400);
}
$response = $client->createMultipartUpload([
'Bucket' => 'the-bucket-name',
'Key' => $key,
'ContentType' => $type,
'Expires' => 60
]);
$mpuKey = !empty($response['Key']) ? $response['Key'] : null;
$mpuUploadId = !empty($response['UploadId']) ? $response['UploadId'] : null;
if (!$mpuKey || !$mpuUploadId) {
return response()->json(['error' => 'Unable to process upload request.'], 400);
}
return response()->json([
'key' => $mpuKey,
'uploadId' => $mpuUploadId
]);
}
public function getUploadedParts($uploadId)
{
// Haven't configured this route yet as I haven't made it this far.
return $uploadId;
}
public function signPartUpload(Request $request, $uploadId, $partNumber)
{
$client = new S3Client([
'version' => 'latest',
'region' => 'us-east-1',
]);
$key = $request->has('key') ? $request->get('key') : null;
if (!is_string($key)) {
return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
}
if (!intval($partNumber)) {
return response()->json(['error' => 's3: the part number must be a number between 1 and 10000.'], 400);
}
// Creating a presigned URL. I don't think this is correct.
$cmd = $client->getCommand('PutObject', [
'Bucket' => 'the-bucket-name',
'Key' => $key,
'UploadId' => $uploadId,
'PartNumber' => $partNumber,
]);
$response = $client->createPresignedRequest($cmd, '+20 minutes');
$presignedUrl = (string)$response->getUri();
return response()->json(['url' => $presignedUrl]);
}
public function completeMultipartUpload(Request $request, $uploadId)
{
$client = new S3Client([
'version' => 'latest',
'region' => 'us-east-1',
]);
$key = $request->has('key') ? $request->get('key') : null;
$parts = json_decode($request->getContent(), true)['parts'];
if (!is_string($key)) {
return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
}
if (!is_array($parts) || !$this->arePartsValid($parts)) {
return response()->json(['error' => 's3: "parts" must be an array of {ETag, PartNumber} objects.'], 400);
}
// The completeMultipartUpload method fails with the following error.
// "Error executing "CompleteMultipartUpload" on "https://the-bucket-name.s3.amazonaws.com/NewProject.png?uploadId=nlWLdbNgB9zgarpLBXnj17eOIGAmQM_xyBArymtwdM71fhbFvveggDmL6fz4blz.B95TLhMatDvodbMb5p2ZMKqdlLeLFoSW1qcu33aRQTlt6NbiP_dkDO90DFO.pWGH"; AWS HTTP error: Client error: `POST https://the-bucket-name.s3.amazonaws.com/NewProject.png?uploadId=nlWLdbNgB9zgarpLBXnj17eOIGAmQM_xyBArymtwdM71fhbFvveggDmL6fz4blz.B95TLhMatDvodbMb5p2ZMKqdlLeLFoSW1qcu33aRQTlt6NbiP_dkDO90DFO.pWGH` resulted in a `400 Bad Request` response:
// <Error><Code>InvalidPart</Code><Message>One or more of the specified parts could not be found. The part may not have be (truncated...)
// InvalidPart (client): One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's entity tag. - <Error><Code>InvalidPart</Code><Message>One or more of the specified parts could not be found. The part may not have been uploaded, or the specified entity tag may not match the part's en"
$result = $client->completeMultipartUpload([
'Bucket' => 'the-bucket-name',
'Key' => $key,
'UploadId' => $uploadId,
'MultipartUpload' => [
'Parts' => $parts,
],
]);
return response()->json(['location' => $result['location']]);
}
public function abortMultipartUpload($uploadId)
{
// Haven't configured this route yet as I haven't made it this far.
return $uploadId;
}
private function arePartsValid($parts)
{
// Validation for the parts will go here, but returning true for now.
return true;
}
}
我可以上传一个多部分文件,纯PHP/服务器端。尽管如此,对于巨大的文件,这是行不通的,因为我必须等待上传完成在我的服务器上,然后上传到AWS在各个部分。
$s3_client = new S3Client([
'version' => 'latest',
'region' => 'us-east-1',
]);
$bucket = 'the-bucket-name';
$tmp_name = $request->file('file')->getPathname();
$folder = Carbon::now()->format('Y/m/d/');
$filename = pathinfo($request->file('file')->getClientOriginalName(), PATHINFO_FILENAME);
$extension = $extension = pathinfo($request->file('file')->getClientOriginalName(), PATHINFO_EXTENSION);
$timestamp = Carbon::now()->format('H-i-s');
$name = "{$folder}{$filename}_{$timestamp}.{$extension}";
$response = $s3_client->createMultipartUpload([
'Bucket' => $bucket,
'Key' => $name,
]);
$uploadId = $response['UploadId'];
$file = fopen($tmp_name, 'r');
$parts = [];
$partNumber = 1;
while (! feof($file)) {
$result = $s3_client->uploadPart([
'Bucket' => $bucket,
'Key' => $name,
'UploadId' => $uploadId,
'PartNumber' => $partNumber,
'Body' => fread($file, 5 * 1024 * 1024),
]);
$parts[] = [
'PartNumber' => $partNumber++,
'ETag' => $result['ETag'],
];
}
$result = $s3_client->completeMultipartUpload([
'Bucket' => $bucket,
'Key' => $name,
'UploadId' => $uploadId,
'MultipartUpload' => [
'Parts' => $parts,
],
]);
我认为所发生的事情是Uppy正在处理while
循环部分客户端。为了做到这一点,我必须返回一个预先签名的URL Uppy可以使用,但我目前返回的预签名URL是不正确的。
我注意到的一件事是,当我在启动多部分上传时,在启动completeMultipartUpload方法之前,不会将任何文件上传到桶中。然而,如果我逐步通过Uppy上传的部分,这些部分似乎被上传为最后的文件,每个部分只是覆盖前面的部分。然后留给我文件的一个片段,即43.5MB文件的最后3.5MB。
发布于 2021-04-23 14:16:01
以下是我如何能让U皮,Vue,和Laravel在一起玩得很好。
Vue组件:
<template>
<div>
<a id="uppy-trigger" @click="isUppyOpen = !isUppyOpen">Open Uppy</a>
<dashboard-modal
:uppy="uppy"
:open="isUppyOpen"
:props="{trigger: '#uppy-trigger'}"
/>
</div>
</template>
<script>
import Uppy from '@uppy/core'
import AwsS3Multipart from '@uppy/aws-s3-multipart';
import '@uppy/core/dist/style.css';
import '@uppy/dashboard/dist/style.css';
export default {
components: {
'dashboard-modal': DashboardModal,
},
data() {
return {
isUppyOpen: false,
}
},
computed: {
// Uppy Instance
uppy: () => new Uppy({
logger: Uppy.debugLogger
}).use(AwsS3Multipart, {
limit: 4,
companionUrl: 'https://mysite.local/',
}),
},
beforeDestroy () {
this.uppy.close();
},
}
</script>
路由:
// AWS S3 Multipart Upload Routes
Route::name('s3.multipart.')->prefix('s3/multipart')
->group(function () {
Route::post('/', ['as' => 'createMultipartUpload', 'uses' => 'AwsS3MultipartController@createMultipartUpload']);
Route::get('{uploadId}', ['as' => 'getUploadedParts', 'uses' => 'AwsS3MultipartController@getUploadedParts']);
Route::get('{uploadId}/{partNumber}', ['as' => 'signPartUpload', 'uses' => 'AwsS3MultipartController@signPartUpload']);
Route::post('{uploadId}/complete', ['as' => 'completeMultipartUpload', 'uses' => 'AwsS3MultipartController@completeMultipartUpload']);
Route::delete('{uploadId}', ['as' => 'abortMultipartUpload', 'uses' => 'AwsS3MultipartController@abortMultipartUpload']);
});
财务主任:
<?php
namespace App\Http\Controllers;
use Aws\S3\S3Client;
use Carbon\Carbon;
use Exception;
use Illuminate\Http\Request;
class AwsS3MultipartController extends Controller
{
private $bucket;
private $client;
public function __construct()
{
$this->bucket = 'the-name-of-the-bucket';
$this->client = new S3Client([
'version' => 'latest',
'region' => 'us-east-1',
]);
}
/**
* Create/initiate the multipart upload
* @param Request $request
* @return JsonResponse
*/
public function createMultipartUpload(Request $request)
{
// Get the filename and type from request
$filename = $request->has('filename') ? $request->get('filename') : null;
$type = $request->has('type') ? $request->get('type') : null;
// Check filename
if (!is_string($filename)) {
return response()->json(['error' => 's3: filename returned from "getKey" must be a string'], 500);
}
// Check type
if (!is_string($type)) {
return response()->json(['error' => 's3: content type must be a string'], 400);
}
// Set up key equal to YYYY/MM/DD/filename_H-i-s.ext
$fileBaseName = pathinfo($filename, PATHINFO_FILENAME);
$extension = pathinfo($filename, PATHINFO_EXTENSION);
$folder = Carbon::now()->format('Y/m/d/');
$timestamp = Carbon::now()->format('H-i-s');
$key = "{$folder}{$fileBaseName}_{$timestamp}.{$extension}";
// Create/initiate the multipart upload
try {
$response = $this->client->createMultipartUpload([
'Bucket' => $this->bucket,
'Key' => $key,
'ContentType' => $type,
'Expires' => 60
]);
} catch (Exception $e) {
return response()->json(['error' => $e->getMessage()], 400);
}
// Multipart upload key and id
$mpuKey = !empty($response['Key']) ? $response['Key'] : null;
$mpuUploadId = !empty($response['UploadId']) ? $response['UploadId'] : null;
// Check multipart upload key and id
if (!$mpuKey || !$mpuUploadId) {
return response()->json(['error' => 'Unable to process upload request.'], 400);
}
return response()->json([
'key' => $mpuKey,
'uploadId' => $mpuUploadId
]);
}
/**
* Get parts that have been uploaded
* @param Request $request
* @param string $uploadId
* @return JsonResponse
*/
public function getUploadedParts(Request $request, string $uploadId)
{
$key = $request->has('key') ? $request->get('key') : null;
// Check key
if (!is_string($key)) {
return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
}
$parts = [];
$getParts = true;
$startAt = 0;
// Get parts uploaded so far
while ($getParts) {
$partsPage = $this->listPartsPage($key, $uploadId, $startAt, $parts);
if (isset($partsPage['error'])) {
return response()->json(['error' => $partsPage['error']], 400);
}
if ($partsPage['isTruncated']) {
$startAt = $partsPage['nextPartNumberMarker'];
} else {
$getParts = false;
}
}
return response()->json(
$parts,
);
}
/**
* Create a pre-signed URL for parts to be uploaded to
* @param Request $request
* @param string $uploadId
* @param int $partNumber
* @return JsonResponse
*/
public function signPartUpload(Request $request, string $uploadId, int $partNumber)
{
$key = $request->has('key') ? $request->get('key') : null;
// Check key
if (!is_string($key)) {
return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
}
// Check part number
if (!intval($partNumber)) {
return response()->json(['error' => 's3: the part number must be a number between 1 and 10000.'], 400);
}
// Create the upload part command and get the pre-signed URL
try {
$command = $this->client->getCommand('UploadPart', [
'Bucket' => $this->bucket,
'Key' => $key,
'PartNumber' => $partNumber,
'UploadId' => $uploadId,
'Body' => '',
]);
$presignedUrl = $this->client->createPresignedRequest($command, '+20 minutes');
} catch (Exception $e) {
return response()->json(['error' => $e->getMessage()], 400);
}
// Convert the pre-signed URL to a string
$presignedUrlString = (string)$presignedUrl->getUri();
return response()->json(['url' => $presignedUrlString]);
}
/**
* Complete the multipart upload
* @param Request $request
* @param string $uploadId
* @return JsonResponse
*/
public function completeMultipartUpload(Request $request, string $uploadId)
{
$key = $request->has('key') ? $request->get('key') : null;
$parts = json_decode($request->getContent(), true)['parts'];
// Check the key
if (!is_string($key)) {
return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
}
// Check the parts
if (!is_array($parts) || !$this->arePartsValid($parts)) {
return response()->json(['error' => 's3: "parts" must be an array of {ETag, PartNumber} objects.'], 400);
}
// Complete the multipart upload
try {
$result = $this->client->completeMultipartUpload([
'Bucket' => $this->bucket,
'Key' => $key,
'UploadId' => $uploadId,
'MultipartUpload' => [
'Parts' => $parts,
],
]);
} catch (Exception $e) {
return response()->json(['error' => $e->getMessage()], 400);
}
// Change forwardslash entities to forwardslashes
$location = urldecode($result['Location']);
return response()->json(['location' => $location]);
}
public function abortMultipartUpload(Request $request, $uploadId)
{
$key = $request->has('key') ? $request->get('key') : null;
// Check the key
if (!is_string($key)) {
return response()->json(['error' => 's3: the object key must be passed as a query parameter. For example: "?key=abc.jpg"'], 400);
}
// Cancel the multipart upload
try {
$response = $this->client->abortMultipartUpload([
'Bucket' => $this->bucket,
'Key' => $key,
'UploadId' => $uploadId,
]);
} catch (Exception $e) {
//
}
return response()->json();
}
private function listPartsPage(string $key, string $uploadId, int $startAt, array &$parts)
{
// Configure response
$response = [
'isTruncated' => false,
];
// Get list of parts uploaded
try {
$result = $this->client->listParts([
'Bucket' => $this->bucket,
'Key' => $key,
'PartNumberMarker' => $startAt,
'UploadId' => $uploadId,
]);
} catch (Exception $e) {
return ['error' => 's3: unable to continue upload. The upload may have been aborted.'];
}
// Add found parts to parts array
if ($result->hasKey('Parts')) {
array_push($parts, ...$result->get('Parts'));
}
// Check if parts are truncated
if ($result->hasKey('IsTruncated') && $result->get('IsTruncated')) {
$response['isTruncated'] = true;
$response['nextPartNumberMarker'] = $result->get('NextPartNumberMarker');
}
return $response;
}
/**
* Validate the parts for the multipart upload
* @param array $parts An associative array of parts with PartNumber and ETag
* @return bool
*/
private function arePartsValid(array $parts)
{
if (!is_array($parts)) {
return false;
}
foreach ($parts as $part) {
if (!is_int($part['PartNumber']) || !is_string($part['ETag'])) {
return false;
}
}
return true;
}
}
发布于 2021-05-14 04:34:50
您可以使用这个预先制作好的laravel包,通过laravel和uppy轻松地实现多部分上传:
https://github.com/TappNetwork/laravel-uppy-s3-multipart-upload
https://stackoverflow.com/questions/67201655
复制相似问题