将 Team Foundation Service Git 仓库镜像到 GitHub、Bitbucket 或类似的公共 Git 服务





5.00/5 (1投票)
一个自定义生成活动,用于将 TFS Git 仓库镜像到公共 Git 仓库。
引言
微软托管的 Team Foundation Server (TFS) 服务 (tfs.visualstudio.com) 不支持公共仓库。可以使用自定义生成活动将 TFS Git 仓库镜像到公共 Git 服务,如 GitHub (www.github.com) 或 Bitbucket (www.bitbucket.com)。这无缝地提供了对代码库的便捷公共访问,同时保持了向公众发布仓库的控制权。
所需工作知识
- Team Foundation Server 生成流程和活动
- Git
- Visual Studio 2013
目录
资源
原型 Visual Studio 解决方案
1.0. 问题
微软托管的 Team Foundation Server (TFS) 服务不支持公共仓库。
2.0. 要求
- 作为生成过程的一部分,将 TFS Git 仓库镜像到只读的公共 Git 仓库
- 支持 GitHub 和 Bitbucket 的托管 Git 服务
3.0. 背景
Team Foundation Service 支持托管和生成 Git 仓库。生成服务的标准 Git 生成定义 GitTemplate.xaml
使用生成活动 Microsoft.TeamFoundation.Build.Activities.Git.GitPull
来克隆仓库并检出分支。然后构建检出的分支。
在底层,该服务依赖 LibGit2Sharp 与 Git 交互 [1] (github.com/libgit2/libgit2sharp)。"git.exe" 未安装在生成控制器上 [2]。
TFS 用自定义实现替换了标准的 LibGit2Sharp 连接管道,该实现能够对服务进行身份验证。TFS 创建一个自定义的 LibGit2Sharp.SmartSubtrtansport
,它包装了 HttpClient
。HttpClient
又接收一个消息处理程序来处理身份验证(Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler
)。参见图 3-1。
图 3-1. "TfsSmartSubtransport" 的摘录。
//Source: "Microsoft.TeamFoundation.Build.Activities.Git.TfsSmartSubtransport" // in "Microsoft.TeamFoundation.Build.Activities.dll" (Version 12.0.0.0) //Note: "TfsSmartHttpClient" inherits from "HttpClient" private TfsSmartSubtransport.TfsSmartHttpClient BuildHttpClientForUri(Uri serviceUri) { VssCredentials vssCredential = VssCredentials.LoadCachedCredentials( CredentialsStorageRegistryKeywords.Build, serviceUri, false, CredentialPromptType.DoNotPrompt); VssHttpRequestSettings vssHttpRequestSetting = new VssHttpRequestSettings(); vssHttpRequestSetting.ExpectContinue = false; TfsSmartSubtransport.TfsSmartHttpClient tfsSmartHttpClient = new TfsSmartSubtransport.TfsSmartHttpClient( new VssHttpMessageHandler(vssCredential, vssHttpRequestSetting)); tfsSmartHttpClient.DefaultRequestHeaders.Add("User-Agent", string.Concat("git/1.0 (Microsoft Git Client [Team Build] ", TfsSmartSubtransport.s_assemblyVersion, ")")); tfsSmartHttpClient.Timeout = TimeSpan.FromMinutes(30); return tfsSmartHttpClient; }
尽管 Team Foundation Service 支持 Git 仓库,但存在限制。无法通过 Visual Studio 2012 修改 Git 生成定义。目前需要 Visual Studio 2013 来选择不同的生成模板。
该服务支持在团队项目中拥有多个 Git 仓库。但是,当仓库的名称与其他仓库或另一个团队项目相同时,似乎会出现问题。这在生成定义中选择仓库和分支时会成为问题。在以编程方式处理 TFS 时,也会返回不正确的仓库 URL。
对于一个项目中的多个仓库,客户端支持也有限。Visual Studio 2012 和在线门户都只显示主仓库。
4.0. 可能的方法
表面上看,我们应该能够创建一个自定义生成活动,该活动使用 LibGit2Sharp 将 TFS 仓库镜像到另一个服务。不幸的是,我们很快就遇到了问题。
Git 生成定义不包含与传统 Team Foundation Version Control (TFVC) 生成过程相同的生成活动或变量集。没有 "工作区",并且关于编写与 TFS API 交互的自定义生成活动(该活动与 Git 仓库和生成服务交互)的文档非常少,几乎没有。
此外,LibGit2Sharp 库仍在开发中,并且不完整。该库不具备我们所需的所有功能。即使我们能以当前形式使用 LibGit2Sharp 库,TFS 的许多自定义连接管道也是内部或私有的。我们需要依赖反射对不断变化的 codebase 进行操作,这会带来麻烦。
另一种方法是使用 "git.exe" shell 而不是 LibGit2Sharp。虽然 "git.exe" 没有安装在生成控制器上,但没有什么能阻止我们将可执行文件和相关文件上传到 TFS。问题就变成了身份验证。要使此方法生效的唯一方法是在 TFS 中启用 "备用凭据",并将用户名和密码作为生成定义中的值。在这种情况下,直接在 TFS 之外使用 Git 复制仓库会更简单。
另一种方法是使用其他库替换 LibGit2Sharp,并替换连接管道,以便库可以与 TFS 一起使用。唯一真正的替代 .Net 库是 NGit (github.com/mono/ngit),它是 Java JGit 库的移植。NGit 的问题在于,替换连接管道所需的类和方法是内部的或私有的。没有什么能阻止我们添加一个自定义类型来与 TFS 交互,但我们不能继承实现许多底层逻辑的基类。即使我们可以继承这些基类,由于访问权限问题,也有几个必需的嵌套类和实用类型无法访问。
与使用 "git.exe" 的方法一样,我们可以在 TFS 中设置 "备用凭据"。在这种情况下,我们可以不经修改地使用 NGit,并将用户名和密码从生成定义传递到 NGit。但直接在 TFS 之外使用 Git 复制仓库会更简单。
一种类似的方法,尽管不是理想的,是修改 NGit 的可访问性,以便我们可以替换连接管道。NGit 已在 GitHub 上发布,初步审查表明该项目相对稳定。没有技术原因阻止我们简单地全局替换访问修饰符。随着 NGit 的更新,可以重复此过程。修改访问权限造成问题的风险远小于重新创建使用当前形式的库所需的底层功能。
第四种方法是使用 Git hook 来触发镜像。但是,Team Foundation Service 不支持 Git hook [3]。
5.0. 解决方案
在所有可能的方法中,最直接的解决方案是
- 创建一个自定义生成活动,该活动将正在生成的 TFS Git 仓库镜像到公共 Git 服务
- 使用 NGit 库克隆并推送正在生成的 Git 仓库的镜像副本到公共 Git 服务
- 修改 NGit 库的访问权限,允许我们替换连接管道
- 使用 TFS 替换 LibGit2Sharp 连接管道的相同方法,替换 NGit 连接管道,以便库可以连接到 TFS
5.1. 原型
创建生成活动是直接的,并且有很好的文档记录。但我们很快就遇到了一个问题,即如何使用 TFS API 处理 Git 仓库。关于如何在生成活动中使用 Git 的文档非常少,几乎没有。我们无法使用 Workstation.Current.GetLocalWorkspaceInfo(...
来创建对服务器的引用,并且生成定义没有与传统 TFVC 生成模板相同的变量集。
最终,生成活动上下文为我们提供了对 TFS 服务器的访问权限。参见图 5-1。
图 5-1. Git 仓库生成活动中的 TFS Api。
...
//Evaluate build environment
IBuildDetail BuildDetail = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<IBuildDetail>(Context, WellKnownEnvironmentVariables.BuildDetail);
IBuildDefinitionSourceProvider DefaultSourceProvider = BuildDetail.BuildDefinition.GetDefaultSourceProvider();
string RepositoryUrl = "";
string RepositoryName = "";
if (BuildSourceProviders.IsGit(DefaultSourceProvider.Name) == true)
{
RepositoryUrl = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryUrl);
RepositoryName = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryName);
}
string SourcesDirectory = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<string>(Context, WellKnownEnvironmentVariables.SourcesDirectory);
TfsTeamProjectCollection TFSServer = BuildDetail.BuildServer.TeamProjectCollection;
string TeamProject = BuildDetail.TeamProject;
//8-7-2013: The "BuildSourceProviders.GitProperties.RepositoryUrl"
// value is not correct when there are multiple Git repositories in a team project.
//The returned url is in the format of "https://tenant.visualstudio.com/
// DefaultCollection/_git/{Repository}" and does not include the team project
//name. The required format is "https://tenant.visualstudio.com/
// DefaultCollection/{TeamProject}/_git/{Repository}".
//8-8-2013 Note: We run into a type load exception when using
//"TFSServerGitService.QueryRepositories(..." on Team Foundation Service.
//GitRepositoryService TFSServerGitService = TFSServer.GetService<GitRepositoryService>();
//GitRepository CurrentRepository = null;
//foreach (GitRepository Repository in TFSServerGitService.QueryRepositories(""))
//{
// if (Repository.Name == RepositoryName && Repository.ProjectReference.Name == TeamProject)
// {
// CurrentRepository = Repository;
// break;
// }
//}
//if(CurrentRepository != null)
//{
// RepositoryUrl = CurrentRepository.RemoteUrl;
//}
if (TeamProject != RepositoryName)
{
//Format: "https://tenant.visualstudio.com/DefaultCollection/{TeamProject}/_git/{Repository}"
RepositoryUrl = TFSServer.Uri.ToString() + "/" + TeamProject + "/_git/" + RepositoryName;
}
...
NGit 通过 "传输协议" 处理到 Git 仓库的连接。实际上,NGit 会解析 URI 并找到匹配的协议类型。然后,协议类型会创建一个知道如何处理仓库的传输对象。
NGit HTTP 传输创建一个简单的连接对象,该对象使用 HttpWebRequest
调用 Git 服务器。每次调用都会创建一个新的连接对象。
我们可以通过创建一个继承自 NGit HTTP 传输类型 NGit.Transport.TransportHttp
的自定义传输类型来实现自己的传输。然后,我们可以重写创建连接对象的方法,并返回自己的连接类型。我们的连接类型可以继承 NGit HTTP 连接类型 Sharpen.URLConnection
,并将 HttpWebRequest
的使用替换为 HttpClient
。这允许我们传入 VssHttpMessageHandler
来处理身份验证,方式与 TFS 使用 LibGit2Sharp 相同。
图 5-2 显示了一个概念验证生成活动。输出 Git 仓库的列表通过 OutputRepositoryUrls
参数传入。URL 格式为 https://USERNAME:PASSWORD@domain.org/abc/efg.git。
生成活动的一般工作流程是
- 创建对当前服务器和生成过程的引用
- 清除已加载的 NGit 传输协议
- 添加自定义传输协议(图 5-3)
- 创建正在生成的 TFS Git 仓库的 "裸克隆"("bare clone")
- 删除自定义传输协议并恢复原始已加载的 NGit 传输协议
- 将 "裸克隆" 的镜像副本推送到每个输出仓库
- 删除 "裸克隆" 仓库副本
图 5-3 显示了自定义传输协议。该协议返回一个自定义传输类型(图 5-4),该类型又返回一个自定义连接对象(图 5-5)。
自定义连接对象使用 HttpClient
而不是 HttpWebRequest
。这会导致 NGit 在发送消息之前关闭输出流的问题。NGit 使用 HttpWebRequest.GetRequestStream()
来写入输出,然后在写入消息后关闭流。这迫使我们使用一个无法被 NGit 关闭的自定义流对象(图 5-6)。
图 5-2. "MirrorRepository" TFS 生成活动。
using Microsoft.TeamFoundation;
using Microsoft.TeamFoundation.Build.Activities.Extensions;
using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Build.Common;
using Microsoft.TeamFoundation.Build.Workflow.Activities;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Git.Client;
using Microsoft.TeamFoundation.Git.Common;
using Microsoft.TeamFoundation.VersionControl.Client;
using Microsoft.VisualStudio.Services.Common;
using System;
using System.Activities;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.ExceptionServices;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git.Activities
{
[Microsoft.TeamFoundation.Build.Client.BuildActivity(
Microsoft.TeamFoundation.Build.Client.HostEnvironmentOption.All)]
[Microsoft.TeamFoundation.Build.Client.BuildExtension(
Microsoft.TeamFoundation.Build.Client.HostEnvironmentOption.All)]
public class MirrorRepository : System.Activities.AsyncCodeActivity
{
#region Private Variables
#endregion
#region Constructors
public MirrorRepository()
{
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
//Note: "RepositoryUrl" is in the format
//of https://USERNAME:PASSWORD@domain.org/abc/efg.git
[System.Activities.RequiredArgument]
public System.Activities.InArgument<IEnumerable<string>>
OutputRepositoryUrls { get; set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
#endregion
#region Private Methods
protected sealed override IAsyncResult BeginExecute(
AsyncCodeActivityContext Context, AsyncCallback Callback, object State)
{
//See: http://stackoverflow.com/questions/16960312/
// implementing-asynccodeactivities-using-c-sharp-async-await
System.Threading.Tasks.Task ExecutionTask = this.ExecuteAsync(Context);
System.Threading.Tasks.TaskCompletionSource<object> TCS =
new System.Threading.Tasks.TaskCompletionSource<object>(State);
ExecutionTask.ContinueWith(T =>
{
if (T.IsFaulted)
{
TCS.TrySetException(T.Exception.InnerExceptions);
}
else if (T.IsCanceled)
{
TCS.TrySetCanceled();
}
else
{
TCS.TrySetResult(null);
}
if (Callback != null)
{
Callback(TCS.Task);
}
});
return TCS.Task;
}
protected sealed override void EndExecute(AsyncCodeActivityContext Context, IAsyncResult Result)
{
System.Threading.Tasks.Task ExecutionTask = (Task)Result;
}
protected async Task ExecuteAsync(AsyncCodeActivityContext Context)
{
//8-4-2013 Note: TFS uses LibGit2Sharp (https://github.com/libgit2/libgit2sharp) to interact with Git.
//TFS creates a "LibGit2Sharp.SmartSubtrtansport" which wraps aroung an HttpClient that is passed a
//message handler for authentication (See: "Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler"
//and "Microsoft.VisualStudio.Services.Common.VssCredentials").
//See: "Microsoft.TeamFoundation.Build.Activities.Git.TfsSmartSubtransport.BuildHttpClientForUri(..."
//See: "Microsoft.TeamFoundation.Build.Activities.Git.GitPull" TFS activity. Note nested classes
//"GitClone" and "GitFetch".
//LibGit2Sharp is incomplete and does not appear to handle the required Git commands.
//NGit (https://github.com/mono/ngit) is more complete, but much of the library is marked internal
//or private. The library does not use HttpClient (which is needed to use "VssHttpMessageHandler"
//for authentication) and implementing a custom transport protocol requires rewriting a great deal
//of low level implementation that cannot be used because of the internal protection.
//As a stopgap we removed the internal protections in the NGit library. This allows us to
//swap out the http connection class and use HttpClient.
//Once LibGit2Sharp is more complete, the modified NGit library should be replaced with LibGit2Sharp.
//Note: TFS as part of a build clones and checks out a single Git branch. We must create a "bare clone"
//of the entire repository to then push a mirror of our copy to the output repository.
List<System.IO.DirectoryInfo> CleanupDirectories = new List<System.IO.DirectoryInfo>();
try
{
List<Uri> OutputRepositories = new List<Uri>();
foreach (string OutputRepositoryUrl in this.OutputRepositoryUrls.Get(Context))
{
try
{
if (String.IsNullOrWhiteSpace(OutputRepositoryUrl) == false)
{
OutputRepositories.Add(new Uri(OutputRepositoryUrl));
}
}
catch { }
}
if (OutputRepositories.Count() > 0)
{
//Evaluate build environment
IBuildDetail BuildDetail = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<IBuildDetail>(Context, WellKnownEnvironmentVariables.BuildDetail);
IBuildDefinitionSourceProvider DefaultSourceProvider =
BuildDetail.BuildDefinition.GetDefaultSourceProvider();
string RepositoryUrl = "";
string RepositoryName = "";
if (BuildSourceProviders.IsGit(DefaultSourceProvider.Name) == true)
{
RepositoryUrl = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryUrl);
RepositoryName = BuildSourceProviders.GetProperty(DefaultSourceProvider.Fields,
BuildSourceProviders.GitProperties.RepositoryName);
}
string SourcesDirectory = Context.GetExtension<IEnvironmentVariableExtension>().
GetEnvironmentVariable<string>(Context, WellKnownEnvironmentVariables.SourcesDirectory);
TfsTeamProjectCollection TFSServer = BuildDetail.BuildServer.TeamProjectCollection;
string TeamProject = BuildDetail.TeamProject;
//8-7-2013: The "BuildSourceProviders.GitProperties.RepositoryUrl"
//value is not correct when there are multiple Git repositories in a team project.
//The returned url is in the format of "https://tenant.visualstudio.com/
// DefaultCollection/_git/{Repository}" and does not include the team project
//name. The required format is "https://tenant.visualstudio.com/
// DefaultCollection/{TeamProject}/_git/{Repository}".
//8-8-2013 Note: We run into a type load exception when using
//"TFSServerGitService.QueryRepositories(..." on Team Foundation Service.
//GitRepositoryService TFSServerGitService = TFSServer.GetService<GitRepositoryService>();
//GitRepository CurrentRepository = null;
//foreach (GitRepository Repository in TFSServerGitService.QueryRepositories(""))
//{
// if (Repository.Name == RepositoryName && Repository.ProjectReference.Name == TeamProject)
// {
// CurrentRepository = Repository;
// break;
// }
//}
//if(CurrentRepository != null)
//{
// RepositoryUrl = CurrentRepository.RemoteUrl;
//}
if (TeamProject != RepositoryName)
{
//Format: "https://tenant.visualstudio.com/DefaultCollection/{TeamProject}/_git/{Repository}"
RepositoryUrl = TFSServer.Uri.ToString() + "/" + TeamProject + "/_git/" + RepositoryName;
}
//Unregister NGit Transport Protocols
List<NGit.Transport.TransportProtocol> NGitTransportProtocols =
new List<NGit.Transport.TransportProtocol>();
try
{
NGitTransportProtocols.AddRange(NGit.Transport.Transport.GetTransportProtocols());
}
catch { }
foreach (NGit.Transport.TransportProtocol T in NGitTransportProtocols)
{
NGit.Transport.Transport.Unregister(T);
}
//Register TFS Transport Protocol
NGit.Transport.TransportProtocol TFSTransportProtocol =
new Adapt.Build.Server.TFS.Git.TFSTransportProtocol(
new Func<Uri, VssCredentials>(
(EndPoint) =>
{
//See: "Microsoft.TeamFoundation.Build.Activities.
// Git.TfsSmartSubtransport.BuildHttpClientForUri(..."
VssCredentials vssCredential = VssCredentials.LoadCachedCredentials(
CredentialsStorageRegistryKeywords.Build,
EndPoint,
false,
CredentialPromptType.DoNotPrompt);
return vssCredential;
}));
NGit.Transport.Transport.Register(TFSTransportProtocol);
//Duplicate Repository
//See: https://help.github.com/articles/duplicating-a-repository
Uri SourceRepositoryUrl = new Uri(RepositoryUrl, UriKind.RelativeOrAbsolute);
Context.TrackBuildMessage("Source Repository Url: " +
RepositoryUrl, BuildMessageImportance.High);
//Create temporary directory to store repository copy
System.IO.DirectoryInfo SourceRepositoryGitDirectory = new System.IO.DirectoryInfo(
System.IO.Path.Combine(System.IO.Path.GetTempPath(),
Guid.NewGuid().ToString() + "\\"));
try
{
if (SourceRepositoryGitDirectory.Exists == false)
{
SourceRepositoryGitDirectory.Create();
}
}
catch { }
//Mark temporary directory for cleanup
CleanupDirectories.Add(SourceRepositoryGitDirectory);
bool IsCloneSuccess = false;
try
{
//Create "bare clone" copy of TFS Git repository
NGit.Api.CloneCommand Command = new NGit.Api.CloneCommand();
Command.SetURI(SourceRepositoryUrl.ToString());
Command.SetDirectory(SourceRepositoryGitDirectory.FullName);
Command.SetCloneAllBranches(true);
Command.SetNoCheckout(true);
Command.SetBare(true);
Command.Call(); //throws exception if fails
IsCloneSuccess = true;
}
catch (Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
//Restore NGit Transport Protocols
NGit.Transport.Transport.Unregister(TFSTransportProtocol);
foreach (NGit.Transport.TransportProtocol T in NGitTransportProtocols)
{
NGit.Transport.Transport.Register(T);
}
if (IsCloneSuccess == true)
{
foreach (Uri OutputRepository in OutputRepositories)
{
string OutputRepositoryLogSafeUrl = OutputRepository.ToString();
if (OutputRepositoryLogSafeUrl.Contains('@') == true)
{
//Remove credentials
OutputRepositoryLogSafeUrl = OutputRepository.Scheme +
"://" + OutputRepositoryLogSafeUrl.Split('@')[1];
}
Context.TrackBuildMessage("Output Repository Url: " +
OutputRepositoryLogSafeUrl, BuildMessageImportance.High);
//Duplicate Repository ("push mirror")
MirrorRepository.TryToGitPushMirror(SourceRepositoryGitDirectory, OutputRepository, Context);
}
}
}
else
{
Context.TrackBuildError("Output Repository Url is not provided.");
}
}
catch(Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
foreach (System.IO.DirectoryInfo CleanupDirectory in CleanupDirectories)
{
try
{
CleanupDirectory.Delete();
}
catch { }
}
}
private static IEnumerable<NGit.Transport.PushResult> TryToGitPushMirror(
System.IO.DirectoryInfo SourceRepositoryGitDirectory,
Uri OutputRepositoryUrl,
AsyncCodeActivityContext Context)
{
List<NGit.Transport.PushResult> Results = new List<NGit.Transport.PushResult>();
try
{
SourceRepositoryGitDirectory.Refresh();
if (SourceRepositoryGitDirectory.Exists == true)
{
NGit.Repository SourceRepository =
new NGit.Storage.File.FileRepository(SourceRepositoryGitDirectory.FullName);
try
{
SourceRepository.Create();
}
catch { }
NGit.Transport.URIish OutputRepositoryURIish = new NGit.Transport.URIish(OutputRepositoryUrl);
NGit.Storage.File.FileBasedConfig SourceRepositoryConfig =
((NGit.Storage.File.FileBasedConfig)SourceRepository.GetConfig());
//Read configuration file content so it can be restored
string ConfigurationFile = SourceRepositoryConfig.GetFile();
string SourceRepositoryConfigFileContent = System.IO.File.ReadAllText(ConfigurationFile);
string OutputRepositoryRemoteName =
System.Guid.NewGuid().ToString().Replace("-", "");
NGit.Transport.RemoteConfig OutputRepositoryRemoteConfig =
new NGit.Transport.RemoteConfig(SourceRepositoryConfig, OutputRepositoryRemoteName);
OutputRepositoryRemoteConfig.IsMirror = true;
OutputRepositoryRemoteConfig.AddURI(OutputRepositoryURIish);
OutputRepositoryRemoteConfig.Update(SourceRepositoryConfig);
SourceRepositoryConfig.Save();
try
{
NGit.Api.Git SourceRepositoryGit = new NGit.Api.Git(SourceRepository);
NGit.Api.PushCommand PushMirrorCommand = SourceRepositoryGit.Push();
PushMirrorCommand.SetRemote(OutputRepositoryRemoteName);
PushMirrorCommand.SetForce(true);
PushMirrorCommand.Add("+refs/*:refs/*");
//--mirror See: https://github.com/libgit2/libgit2/issues/1142
//Command.SetPushAll(); //"refs/heads/*:refs/heads/*"
Sharpen.Iterable<NGit.Transport.PushResult> CommandResults =
PushMirrorCommand.Call();
try
{
Results.AddRange(CommandResults);
}
catch { }
}
catch (Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
//Restore Configuration
System.IO.File.WriteAllText(ConfigurationFile, SourceRepositoryConfigFileContent);
}
}
catch (Exception e)
{
Context.TrackBuildError(e.Message + " " + e.StackTrace);
}
return Results;
}
#endregion
#region Overridden Methods
#endregion
}
}
图 5-3. "TFSTransportProtocol" NGit 传输协议。
using Microsoft.VisualStudio.Services.Common;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class TFSTransportProtocol : NGit.Transport.TransportProtocol
{
#region Private Variables
#endregion
#region Constructors
public TFSTransportProtocol(Func<Uri,VssCredentials> GetCredentials)
{
this.GetCredentials = GetCredentials;
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
protected Func<Uri,VssCredentials> GetCredentials { get; private set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
public override string GetName()
{
return this.GetType().Name;
}
public override ICollection<string> GetSchemes()
{
List<string> Schema = new List<string>();
Schema.Add("http");
Schema.Add("https");
return Schema;
}
public override ICollection<NGit.Transport.TransportProtocol.URIishField> GetRequiredFields()
{
List<NGit.Transport.TransportProtocol.URIishField> Fields =
new List<NGit.Transport.TransportProtocol.URIishField>();
Fields.Add(NGit.Transport.TransportProtocol.URIishField.HOST);
Fields.Add(NGit.Transport.TransportProtocol.URIishField.PATH);
return Fields;
}
public override ICollection<NGit.Transport.TransportProtocol.URIishField> GetOptionalFields()
{
List<NGit.Transport.TransportProtocol.URIishField> Fields =
new List<NGit.Transport.TransportProtocol.URIishField>();
Fields.Add(NGit.Transport.TransportProtocol.URIishField.USER);
Fields.Add(NGit.Transport.TransportProtocol.URIishField.PASS);
Fields.Add(NGit.Transport.TransportProtocol.URIishField.PORT);
return Fields;
}
public override int GetDefaultPort()
{
return 80;
}
public override NGit.Transport.Transport Open(NGit.Transport.URIish RemoteRepositoryUrl)
{
return new TFSTransport(
RemoteRepositoryUrl,
new Func<Uri, System.Net.Http.HttpMessageHandler>((EndPoint) =>
{
return GetVssHttpMessageHandler(this.GetCredentials.Invoke(EndPoint));
}));
}
public override NGit.Transport.Transport Open(NGit.Transport.URIish RemoteRepositoryUrl,
NGit.Repository LocalRepository, string RemoteName)
{
return new TFSTransport(
LocalRepository,
RemoteRepositoryUrl,
new Func<Uri, System.Net.Http.HttpMessageHandler>((EndPoint) =>
{
return GetVssHttpMessageHandler(this.GetCredentials.Invoke(EndPoint));
}));
}
#endregion
#region Private Methods
protected static VssHttpMessageHandler GetVssHttpMessageHandler(VssCredentials Credentials)
{
return new VssHttpMessageHandler(Credentials,
new VssHttpRequestSettings() { ExpectContinue = false });
}
#endregion
#region Overridden Methods
#endregion
}
}
图 5-4. "TFSTransport" NGit 传输。
using NGit;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class TFSTransport : NGit.Transport.TransportHttp
{
#region Private Variables
#endregion
#region Constructors
public TFSTransport(
NGit.Repository LocalRepository,
NGit.Transport.URIish RemoteRepositoryUrl,
Func<Uri, System.Net.Http.HttpMessageHandler> GetHttpClientHandlerInstance = null)
: base(LocalRepository, RemoteRepositoryUrl)
{
this.RemoteRepositoryUrl = RemoteRepositoryUrl;
this.GetHttpClientHandlerInstance = GetHttpClientHandlerInstance;
}
public TFSTransport(
NGit.Transport.URIish RemoteRepositoryUrl,
Func<Uri, System.Net.Http.HttpMessageHandler> GetHttpClientHandlerInstance = null)
: base(RemoteRepositoryUrl)
{
this.RemoteRepositoryUrl = RemoteRepositoryUrl;
this.GetHttpClientHandlerInstance = GetHttpClientHandlerInstance;
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
protected NGit.Transport.URIish RemoteRepositoryUrl { get; private set; }
public NGit.Repository LocalRepository
{
get
{
return base.local;
}
}
protected Func<Uri, System.Net.Http.HttpMessageHandler>
GetHttpClientHandlerInstance { get; private set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
#endregion
#region Private Methods
public override Sharpen.HttpURLConnection HttpOpen(string method, Uri u)
{
System.Net.Http.HttpMessageHandler ClientHandler = null;
if (this.GetHttpClientHandlerInstance != null)
{
try
{
ClientHandler = this.GetHttpClientHandlerInstance.Invoke(u);
}
catch { }
}
Sharpen.HttpURLConnection Connection = new HttpTransportConnection(u, ClientHandler);
Connection.SetRequestMethod(method);
Connection.SetUseCaches(false);
Connection.SetRequestProperty(NGit.Util.HttpSupport.HDR_ACCEPT_ENCODING,
NGit.Util.HttpSupport.ENCODING_GZIP);
Connection.SetRequestProperty(NGit.Util.HttpSupport.HDR_PRAGMA, "no-cache");
return Connection;
}
#endregion
#region Overridden Methods
#endregion
}
}
图 5-5. "HttpTransportConnection" NGit HTTP 连接。
using NGit;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class HttpTransportConnection : Sharpen.HttpURLConnection, IDisposable
{
#region Private Variables
private bool disposed = false;
#endregion
#region Constructors
public HttpTransportConnection(
Uri EndPoint,
System.Net.Http.HttpMessageHandler ClientHandler = null)
: base(EndPoint, null)
{
this.EndPoint = EndPoint;
this.ClientHandler = ClientHandler;
this.Request = new System.Net.Http.HttpRequestMessage(System.Net.Http.HttpMethod.Get, EndPoint);
this.RequestContentStream = new TFSTransportStream();
this.Request.Content = new System.Net.Http.StreamContent(this.RequestContentStream);
if (this.ClientHandler != null)
{
this.Client = new System.Net.Http.HttpClient(this.ClientHandler);
}
else
{
this.Client = new System.Net.Http.HttpClient();
}
this.Response = new Lazy<System.Net.Http.HttpResponseMessage>(() =>
{
//Note: NGit connection object is only used once for a single server call
if(this.Request.Method != System.Net.Http.HttpMethod.Post &&
this.Request.Method != System.Net.Http.HttpMethod.Put )
{
this.Request.Content = null;
}
this.RequestContentStream.Position = 0;
System.Net.Http.HttpResponseMessage Response = this.Client.SendAsync(this.Request).Result;
this.RequestContentStream.IsComplete = true;
this.RequestContentStream.Close();
return Response;
}, true);
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
private Uri EndPoint { get; set; }
private System.Net.Http.HttpClient Client { get; set; }
private System.Net.Http.HttpMessageHandler ClientHandler { get; set; }
private System.Net.Http.HttpRequestMessage Request { get; set; }
private TFSTransportStream RequestContentStream { get; set; }
private Lazy<System.Net.Http.HttpResponseMessage> Response { get; set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
public override void SetUseCaches(bool u)
{
}
public override void SetRequestMethod(string method)
{
this.Request.Method = new System.Net.Http.HttpMethod(method);
}
public override string GetRequestMethod()
{
return this.Request.Method.Method;
}
public override void SetInstanceFollowRedirects(bool redirects)
{
}
public override void SetDoOutput(bool dooutput)
{
}
public override void SetFixedLengthStreamingMode(int len)
{
}
public override void SetChunkedStreamingMode(int n)
{
}
public override void SetRequestProperty(string key, string value)
{
switch(key.ToLowerInvariant())
{
case "content-encoding":
this.Request.Content.Headers.ContentEncoding.Add(value);
break;
case "content-length":
long ContentLength = 0;
if (long.TryParse(value, out ContentLength) == true)
{
this.Request.Content.Headers.ContentLength = ContentLength;
}
break;
case "content-type":
this.Request.Content.Headers.ContentType =
new System.Net.Http.Headers.MediaTypeHeaderValue(value);
break;
case "transfer-encoding":
this.Request.Content.Headers.Add("transfer-encoding", value);
break;
default:
this.Request.Headers.Add(key, value);
break;
}
}
public override string GetResponseMessage()
{
return this.Response.Value.ReasonPhrase;
}
public override void SetConnectTimeout(int ms)
{
}
public override void SetReadTimeout(int ms)
{
}
public override Sharpen.InputStream GetInputStream()
{
return this.Response.Value.Content.ReadAsStreamAsync().Result;
}
public override Sharpen.OutputStream GetOutputStream()
{
return this.RequestContentStream;
}
public override string GetHeaderField(string header)
{
string Value = "";
switch (header.ToLowerInvariant())
{
case "content-encoding":
Value = Value = String.Join(", ",
this.Response.Value.Content.Headers.ContentEncoding);
break;
case "content-length":
Value = this.Response.Value.Content.Headers.ContentLength.ToString();
break;
case "content-type":
Value = this.Response.Value.Content.Headers.ContentType.MediaType;
break;
default:
if (this.Response.Value.Headers.Select(x => x.Key).Contains(header) == true)
{
Value = String.Join(", ", this.Response.Value.Headers.GetValues(header));
}
else if (this.Response.Value.Content.Headers.Select(x => x.Key).Contains(header) == true)
{
Value = String.Join(", ", this.Response.Value.Content.Headers.GetValues(header));
}
break;
}
return Value;
}
public override string GetContentType()
{
return this.Response.Value.Content.Headers.ContentType.MediaType;
}
public override int GetContentLength()
{
return (int)this.Response.Value.Content.Headers.ContentLength.Value;
}
public override int GetResponseCode()
{
return (int)this.Response.Value.StatusCode;
}
public override Uri GetURL()
{
return this.EndPoint;
}
//IDisposable
public virtual void OnDisposing()
{
try
{
this.ClientHandler.Dispose();
this.ClientHandler = null;
}
catch { }
try
{
this.Client.Dispose();
this.Client = null;
}
catch { }
}
#endregion
#region Private Methods
//IDisposable
public void Dispose()
{
this.Dispose(true);
GC.SuppressFinalize(this);
}
protected void Dispose(bool disposing)
{
if (!this.disposed)
{
if (disposing)
{
try
{
this.OnDisposing();
}
catch { }
}
this.disposed = true;
}
}
~HttpTransportConnection()
{
this.Dispose(true); //this.Dispose(false);
}
#endregion
#region Overridden Methods
#endregion
}
}
图 5-6. "TFSTransportStream" 流。
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Adapt.Build.Server.TFS.Git
{
public class TFSTransportStream : System.IO.MemoryStream
{
#region Private Variables
#endregion
#region Constructors
public TFSTransportStream()
{
this.IsComplete = false;
}
#endregion
#region Constant Variables and Enumerations
#endregion
#region Variable Containers
public bool IsComplete { get; set; }
#endregion
#region System Event Handlers
#endregion
#region Custom Event Handlers
#endregion
#region Overridden Event Handlers
#endregion
#region Public Methods
public override void Close()
{
if (this.IsComplete == true)
{
base.Close();
}
}
#endregion
#region Private Methods
#endregion
#region Overridden Methods
#endregion
}
}
5.2. 下载
完整的概念验证 Visual Studio 解决方案可在 资源 下下载。样本项目中包含 NGit 的修改版本,该版本开放了可访问性,允许继承类型。修改后的版本将作为 NuGet 包发布,源代码也将发布。
5.3. 已知问题
1. 持续集成生成触发器
Team Foundation Service 对 Git 仓库的支持有限。在使用自定义 Git 生成过程模板进行自动触发时似乎存在问题。默认模板与持续集成触发器一起工作,但自定义模板会抛出异常。该异常似乎与生成初始化有关。即使使用未修改的默认模板副本,也会抛出异常。
TF215097: An error occurred while initializing a build for build definition
Exception Message: TF10159:
The label name 'G:refs/heads/master:CB8ED4F70E06E1519FBF205C0D7254244DD7AA0B' is not supported.
手动排队生成时不会抛出异常。
6.0. 尽职调查和关键风险
原型可能存在的问题包括
- 作者对 Git 的内部工作原理了解有限。镜像实现可能并非在所有情况下都有效。
- 用于防止 NGit 在发送输出流之前关闭输出流的流对象,仅继承自
System.IO.MemoryStream
并重写了Close()
方法。这个简单的实现可能会导致问题。 - 该原型尚未在不同的生产环境中进行广泛的现场测试。
7.0. 结束语
- 参见:Microsoft.TeamFoundation.Build.Activities.dll 中的
Microsoft.TeamFoundation.Build.Activities.Git
命名空间(版本 12.0.0.0) - 参见:http://tfs.visualstudio.com/en-us/learn/hosted-build-controller-in-vs.aspx
- 参见:http://social.msdn.microsoft.com/Forums/vstudio/en-US/84fea5b9-de95-4157-bb70-5333f86a8eb4/git-commit-hook-stream-available