In work I’ve discovered a number of problems when accessing open source code repositories that are using SVN or GIT. Attempts to access a number of repositories just fails with a connection failed message.
After some time of getting these, and finding some repositories working, I noticed that the common attribute for those repo’s that were accessible, was that they were via http. Now that might be fine, but there is a reason for the repositories to have their own protocols, in that they are tuned to be more efficient at transferring the necessary information. In addition not all sites make the repositories available over http in additional to the native format.
Obviously the core problem here is with the corporate firewall blocking outbound requests over certain ports. Perfectly understandable, but sometimes just a little unhelpful when you need to be able to access various upstream repositories when working with open source to check for bug fixes in the code.
After several problems in accessing some git repos that didn’t have http equivalents I did some searching and found the following website http://www.emilsit.net/blog/archives/how-to-use-the-git-protocol-through-a-http-connect-proxy/ which described how you could use socat to create a tunnel that allowed access to git repositories without having to find one that permitted access over http.
#!/bin/sh
# Use socat to proxy git through an HTTP CONNECT firewall.
# Useful if you are trying to clone git:// from inside a company.
# Requires that the proxy allows CONNECT to port 9418.
#
# Save this file as gitproxy somewhere in your path (e.g., ~/bin) and then run
# chmod +x gitproxy
# git config --global core.gitproxy gitproxy
#
# More details at http://tinyurl.com/8xvpny
# Configuration. Common proxy ports are 3128, 8123, 8000.
_proxy=proxy.yourcompany.com
_proxyport=3128
exec socat STDIO PROXY:$_proxy:$1:$2,proxyport=$_proxyport
After some problems accessing various SVN repositories as well, I managed to come up with a script that was a little more flexible. Note that I’m aware that SVN is supposed to provide that ability to use http proxies, however for some reason it just doesn’t seem to work as intended for me.
#!/bin/bash # # Use socat to proxy common SCM protocols through HTTP CONNECT firewall. # Useful if you are trying to use svn or git where a http version of the repo # is not available for use. For svn, while it supports setting a http proxy in # the config file, I've not found it particularly reliable. # # # Save this file as repoproxy somewhere in your path (e.g., ~/bin), make it executable, # and then create the symlinks either gitproxy or svnproxy in the same location and # following the instructions appropriate to them. # # GIT # # gitproxy # git config --global core.gitproxy gitproxy # # More details and original at # http://www.emilsit.net/blog/archives/how-to-use-the-git-protocol-through-a-http-connect-proxy/ # # SVN # # svnproxy # edit ~/.subversion/config and add "socat = <path to script>" under the tunnels section # # then whenever using svn, simply change the line "svn://" to read "svn+socat://" # and continue to use as normal # # Need proxy settings [[ -z "${http_proxy}" ]] && \ echo "Need http_proxy to be set somewhere in order for this script be able to use socat!" && \ exit 2 _proxy=${http_proxy%:*} _proxyport= if [[ "${http_proxy#*:}" != "${_proxy}" ]] then _proxyport=${http_proxy#*:} fi case ${0##*/} in "svnproxy") targetserver="${1#*@}" targetport="3690" ;; "gitproxy") targetserver=$1 targetport=$2 ;; *) echo "Unknown usage! ${0##*/} is not a recognised command" exit 1 ;; esac echo "socat STDIO PROXY:${_proxy}:${targetserver}:${targetport}${_proxyport:+,proxyport=${_proxyport}}" >&2 exec socat STDIO PROXY:${_proxy}:${targetserver}:${targetport}${_proxyport:+,proxyport=${_proxyport}}
Hopefully this will be useful to people other than just me