WCF将自己托管的服务上的大数据(500MB / 1GB)流式传输如何实现?

内容来源于 Stack Overflow,并遵循CC BY-SA 3.0许可协议进行翻译与使用

  • 回答 (1)
  • 关注 (0)
  • 查看 (102)

我目前遇到了尝试使用WCF自托管服务(无IIS)发送大量数据的问题。使用流式处理结果传输500MB与我的服务与System.OutOfMemoryException崩溃。是否有可能转移这么多的数据?

这是我的WCF配置:

<system.serviceModel>
<services>
  <service  name="CIService" behaviorConfiguration="CIBehavior">        
    <host>
      <baseAddresses>
        <add baseAddress="net.tcp://localhost:6547/CIService/CIService.svc" />
      </baseAddresses>
    </host>
    <endpoint binding="netTcpBinding" 
        bindingConfiguration="netTcpBindingConfig" 
        behaviorConfiguration="CIBehavior.EndpointBehavior" 
        contract="CIService.ICreatable" />
    <endpoint address="mex" 
        binding="mexHttpBinding" 
        name="mexTcpBinding" 
        contract="IMetadataExchange" />
  </service>
</services>
<serviceHostingEnvironment multippleSiteBindingEnabled="True" />
<bindings>
  <netTcpBinding>
    <binding name="netTcpBindingConfig" closeTimeout="00:01:00" openTimeout="00:01:00" 
        receiveTimeout="01:00:00" sendTimeout="00:10:00" 
        hostNameComparisonMode="StrongWildcard" listenBacklog="10" maxConnections="10"
        maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" 
        transferMode="Streamed">
      <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647"
        maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
    </binding>
  </netTcpBinding>
</bindings>

<behaviors>
  <serviceBehaviors>
    <behavior name="CIBehavior">
      <serviceMetadata httpGetEnabled="False" />
      <serviceDebug includeExceptionDetailInFaults="true" />
      <serviceThrottling maxConcurrentCalls="200"  maxConcurrentInstances="2147483647" maxConcurrentSessions="100" />
      <dataContractSerializer maxItemsInObjectGraph="2147483647"/>
    </behavior>
  </serviceBehaviors>
  <endpointBehavior>
    <behavior name="CIBehavior.EndpointBehavior">
      <dataContractSerializer maxItemsInObjectGraph="2147483647" />
    </behavior>
  </endpointBehavior>
</behaviors>
</system.serviceModel>

我的客户端配置:

<system.serviceModel>
<bindings>
  <netTcpBinding>
    <binding name="NetTcpBinding_ICreatable" 
             closeTimeout="00:01:00" openTimeout="00:01:00" 
             receiveTimeout="01:00:00" sendTimeout="00:10:00" 
             transactionFlow="false" 
             transferMode="Streamed" 
             transactionProtocol="OleTransactions" 
             hostNameComparisonMode="StrongWildcard" 
             listenBacklog="10" 
             maxBufferPoolSize="2147483647" 
             maxBufferSize="2147483647" 
             maxConnections="10"
             maxReceivedMessageSize ="2147483647">
      <readerQuotas
        maxDepth="2147483647" 
        maxStringContentLength="2147483647" 
        maxArrayLength="2147483647" 
        maxBytesPerRead="2147483647" 
        maxNameTableCharCount="2147483647" />
      <reliableSession ordered="true" inactivityTimeout="00:10:00" anabled="false" />
    </binding>
  </netTcpBinding>
</bindings>
<client>
  <endpoint name="NetTcpBinding_ICreatable" 
      address="net.tcp://localhost:6547/CIService/CIService.svc" 
      binding="netTcpBinding" 
      bindingConfiguration="NetTcpBinding_ICreatable" 
      behaviorConfiguration="CIBehavior.EndpointBehavior" 
      contract="ICreatable" />
</client>
<behaviors>
  <endpointBehavior>
    <behavior name="CIBehavior.EndpointBehavior">
      <dataContractSerializer maxItemsInObjectGraph="2147483647" />
    </behavior>
  </endpointBehavior>
</behaviors>
</system.serviceModel> 
提问于
用户回答回答于

不需要将maxBufferSizemaxBufferPoolSize设置得如此之高,这可能会导致内存不足异常。默认值应该没问题。

查看MSDN上的Large Data and Streaming特别是大数据特殊安全注意事项部分这段文字很重要

MaxBufferSize属性需要约束WCF缓冲的内存。流式传输时将其设置为安全值(或保持默认值)很重要。例如,假设您的服务必须接收最大4 GB的文件并将它们存储在本地磁盘上。假设你的内存受到限制,一次只能缓冲64KB的数据。然后,您将MaxReceivedMessageSize设置为4 GB,将MaxBufferSize设置为64 KB。另外,在你的服务实现中,你必须确保你只读取来自64-KB数据块的入站流,并且在前一个数据块被写入磁盘并从内存中丢弃之前不读取下一个数据块。

我汇集了一个非常简单的例子,即将数据从自身托管服务传输到控制台客户端。为了保持短小,我只添加了服务器代码和客户端的一部分。

服务合同

using System.IO;
using System.ServiceModel;

namespace Service
{
    [ServiceContract]
    public interface IStream
    {
        [OperationContract]
        Stream GetLargeObject();
    }
}

服务实施

using System;
using System.IO;
using System.ServiceModel;

namespace Service
{
   [ServiceBehavior]
   public class StreamService : IStream
   {
       public Stream GetLargeObject()
       {
           // Add path to a big file, this one is 2.5 gb
           string filePath = Path.Combine(Environment.CurrentDirectory, "C:\\Temp\\BFBC2_PC_Client_R11_795745_Patch.exe");

        try
        {
            FileStream imageFile = File.OpenRead(filePath);
            return imageFile;
        }
        catch (IOException ex)
        {
            Console.WriteLine(String.Format("An exception was thrown while trying to open file {0}", filePath));
            Console.WriteLine("Exception is: ");
            Console.WriteLine(ex.ToString());
            throw;
        }
    }
 }
}

服务主要

using System;
using System.ServiceModel;

namespace Service
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                using (var serviceHost = new ServiceHost(typeof(StreamService)))
                {
                    serviceHost.Open();

                    Console.WriteLine("Press Any Key to end");
                    Console.ReadKey();
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.ToString());
            }
        }
    }
}

服务app.config

<?xml version="1.0"?>
<configuration>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
  </startup>

  <system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name="StreamServiceBehavior">
          <serviceMetadata httpGetEnabled="True" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <bindings>
      <netTcpBinding>
        <binding name="NewBinding0" transferMode="Streamed"/>
      </netTcpBinding>
    </bindings>
    <services>
      <service behaviorConfiguration="StreamServiceBehavior" name="Service.StreamService">
        <endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
          bindingConfiguration="NewBinding0" bindingName="" contract="Service.IStream" />
        <endpoint address="mex" binding="mexHttpBinding"
          contract="IMetadataExchange" />
        <host>
          <baseAddresses>
            <add baseAddress="http://localhost:8080/StreamService" />
          </baseAddresses>
        </host>
      </service>
    </services>
  </system.serviceModel>
</configuration>

启动服务,可能需要在管理员帐户下运行以打开套接字。创建客户端控制台应用程序并使用URL http:// localhost:8080 / StreamService添加服务引用,并使用Service作为生成客户端的名称空间。

客户主

using System;
using System.IO;
using Client.Service;

namespace Client
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                using (StreamClient streamClient = new StreamClient())
                {
                    streamClient.Open();

                    using (FileStream fileStream = new FileStream("c:\\temp\\bigfile.exe",FileMode.Create))
                    {
                        streamClient.GetLargeObject().CopyTo(fileStream);    
                    }
                }

                Console.WriteLine("Press any key to end");
                Console.ReadKey();
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }
}

生成的客户端配置文件需要稍微修改,增加receiveTimeout并将maxReceivedMessageSize设置为“4294967295”

<system.serviceModel>
    <bindings>
        <netTcpBinding>
            <binding name="NetTcpBinding_IStream" closeTimeout="00:01:00"
                openTimeout="00:01:00" receiveTimeout="00:30:00" sendTimeout="00:01:00"
                transactionFlow="false" transferMode="Streamed" transactionProtocol="OleTransactions"
                hostNameComparisonMode="StrongWildcard" listenBacklog="10"
                maxBufferPoolSize="524288" maxBufferSize="65536" maxConnections="10"
                maxReceivedMessageSize="4294967295">
                <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                    maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                <reliableSession ordered="true" inactivityTimeout="00:10:00"
                    enabled="false" />
                <security mode="Transport">
                    <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" />
                    <message clientCredentialType="Windows" />
                </security>
            </binding>
        </netTcpBinding>
    </bindings>
    <client>
        <endpoint address="net.tcp://localhost:9000/streamserver" binding="netTcpBinding"
            bindingConfiguration="NetTcpBinding_IStream" contract="Service.IStream"
            name="NetTcpBinding_IStream">

        </endpoint>
    </client>
</system.serviceModel>

启动服务,然后启动客户端。它会流没有问题的大文件。

扫码关注云+社区