Comments (23)

You may or may not need this anymore, but others undoubtedly will. I ran into the same problem cloning Terminals (http://terminals.codeplex.com/) and found a way to solve it! Set the hgsubversion.stupid to true in your hg config file, and cloning works just fine. I guess SvnBridge isn't fully SVN compatible, but the 'stupid' way of using svn works well enough, although absurdly slow, so I would still leave this bug open, and add my solution as a workaround for the time being.

@dreamkxd Would you mind sharing your steps for getting subvertpy to install correctly?
After building and installing it I first get "No module named hgsubversion" when subvertpy_wrapper tries to load subvertpy. Commented out references to hgsubversion (just for testing), and now I get an error that the subvertpy module cannot be found. I'm fairly sure I'm just doing it all wrong, so I'd rather have a nice and concise how-to for getting this to work.

@dreamkxd Sorry, should've mentioned. I'm using Windows as well. As I mentioned, I did built and installed subvertpy by doing "setup.py build" and "setup.py install". Did the same with your "hgsubversion.svnmeta". The results are as I mention in my previous message. Could you please describe how you got this to work? Thanks!

The line
for p in sorted(paths.keys())
is the place I getting the things to be works.
Maybe you just didn't configure the HG properly to using hgsubversion.svnmeta

importerrnoimporttracebackfrommercurialimportnodefrommercurialimportcontextimportsvnexternalsimportutilclassMissingPlainTextError(Exception):"""Exception raised when the repo lacks a source file required for replaying a txdelta. """classReplayException(Exception):"""Exception raised when you try and commit but the replay encountered an exception. """defupdateexternals(ui,meta,current):ifnotcurrent.externals:return# accumulate externals records for all branchesrevnum=current.rev.revnumbranches={}forpath,entryincurrent.externals.iteritems():subpath,branch_info=meta.split_path(path,revnum)ifnotmeta.is_active_branch(branch_info):ui.warn('WARNING: Invalid path "%s" in externals\n'%path)continuebp=branch_info[0]ifbpnotinbranches:parent=meta.get_parents(branch_info)[2]#assert parent != node.nullid, (branch_info, path, entry)pctx=meta.repo[parent]branches[bp]=(svnexternals.parse(ui,pctx),pctx)branches[bp][0][subpath]=entry# register externals file changesforbp,(external,pctx)inbranches.iteritems():ifbpandbp[-1]!='/':bp+='/'updates=svnexternals.getchanges(ui,meta.repo,pctx,external)forfn,datainupdates.iteritems():path=(bpandbp+fn)orfnifdataisnotNone:current.set(path,data,False,False)else:current.delete(path)defcommit_branch(meta,branch_path,in_files,is_tag):ui=meta.uicurrent=meta.editor.currentr=current.revdate=meta.fixdate(r.date)files=dict(in_files)branch_info=meta.exist_branch(branch_path,r.revnum,2)#Whenever what happened, we must modify on an active path.assertmeta.is_active_branch(branch_info),branch_infocopy_from_parent,direct_parent,real_parent=meta.get_parents(branch_info)parentctx=meta.repo.changectx(real_parent)deffilectxfn(repo,memctx,path):ifpath=='.hgtags':returncontext.memfilectx(path=path,data=files[path],islink=False,isexec=False,copied=None)current_file=files[path]ifcurrent_fileincurrent.deleted:raiseIOError(errno.ENOENT,'%s is deleted'%path)copied=current.copies.get(current_file)flags=parentctx.flags(path)is_exec=current.execfiles.get(current_file,'x'inflags)is_link=current.symlinks.get(current_file,'l'inflags)ifcurrent_fileincurrent.files:data=current.files[current_file]ifis_linkanddata.startswith('link '):data=data[len('link '):]elifis_link:ui.debug('file marked as link, but may contain data: ''%s (%r)\n'%(current_file,flags))else:data=parentctx.filectx(path).data()returncontext.memfilectx(path=path,data=data,islink=is_link,isexec=is_exec,copied=copied)ifbranch_pathinmeta.branch_tags:tagdata=''if'.hgtags'inparentctx:tagdata=parentctx.filectx('.hgtags').data()files['.hgtags']=tagdata+meta.branch_tags[branch_path]is_closed=notmeta.inside_branch_interval(branch_info,r.revnum)extra=meta.genextra(r.revnum,branch_path,is_tagoris_closed)parents=(copy_from_parent,direct_parent)ifcopy_from_parent==node.nullid:parents=(direct_parent,node.nullid)current_ctx=context.memctx(meta.repo,parents,r.messageorutil.default_commit_msg(ui),files.keys(),filectxfn,meta.authors[r.author],date,extra)new_hash=meta.repo.commitctx(current_ctx)util.describe_commit(ui,new_hash,branch_path)assert(r.revnum,branch_path)notinmeta.revmap, \
"Now for each branch_path, revision pair, there is exactly one corresponding revmap record!"meta.revmap[r.revnum,branch_path]=new_hash_=meta.repo.changectx(new_hash)defconvert_rev(ui,meta,svn,r,firstrun):editor=meta.editoreditor.current.clear()editor.current.rev=riffirstrunandmeta.revmap.oldest<=0:# We know nothing about this project, so fetch everything before# trying to apply deltas.ui.debug('replay: fetching full revision\n')svn.get_revision(r.revnum,editor)else:svn.get_replay(r.revnum,editor,meta.revmap.oldest)revnum=r.revnumpaths=r.pathsforpinsorted(paths.keys()):ed_p=p[1:]ifed_pineditor.current.touched:continuesubpath,branch_info=meta.split_path(p,revnum)ifnotmeta.is_active_branch(branch_info):continueaction=paths[p].actionkind=meta.checkpath(p,revnum)msg="fetching %s:%s:%s:%s from %s:%s"%(p,revnum,kind,action,paths[p].copyfrom_path,paths[p].copyfrom_rev)ui.status('Start %s\n'%msg)ifaction=='D'oraction=='R':#First deal with DELETEeditor.delete_entry(ed_p,revnum-1,None)if(kind=='d'):#Because it's lost, so the property should be fetched separately.editor.current.missing_props.add(ed_p+'/')editor.current.findprops(svn,ed_p,ed_p+'/')ifaction=='A'oraction=='R':editor.add_directory(ed_p,None,paths[p].copyfrom_path,paths[p].copyfrom_rev)elifaction=='M':#Only can modify propspasselif(kind=='f'):#It's a file, directly missing itifaction=='A'oraction=='M'oraction=='R':#TODO: Using unified diff to reduce bandwidth, ask dreamkxd@163.comeditor.current.missing.add(ed_p)elifaction!='D':raiseRuntimeError("The kind must be an clear value when %s"%msg)current=editor.currentcurrent.findmissing(svn)updateexternals(ui,meta,current)ifcurrent.exceptionisnotNone:#pragma: no covertraceback.print_exception(*current.exception)raiseReplayException()ifcurrent.missing:raiseMissingPlainTextError()# Periodically generate the list of files to commitfiles_to_commit=set(current.files.keys())files_to_commit.update(current.symlinks.keys())files_to_commit.update(current.execfiles.keys())files_to_commit.update(current.deleted.keys())# back to a list and sort so we get sane behaviorfiles_to_commit=list(files_to_commit)files_to_commit.sort()all_changed=set.union(editor.current.changed,meta.changed)tag_added=[tfortinmeta.addedifmeta.is_tag(t,r.revnum)]tag_changed=[tfortinall_changedifmeta.is_tag(t,r.revnum)]tag_deleted=[tfortinmeta.deletedifmeta.is_tag(t,r.revnum-1)]tag_changes=set(tag_added+tag_changed+tag_deleted)iftag_changes:ui.status("[r%s] !Changed tags:%s\n"%(r.revnum,tag_changes))ifmeta.added:ui.status("[r%s] !Added paths:%s\n"%(r.revnum,meta.added))ifall_changed:ui.status("[r%s] !Changed paths:%s\n"%(r.revnum,all_changed))ifmeta.deleted:ui.status("[r%s] !Deleted paths:%s\n"%(r.revnum,meta.deleted))all_modifyed=set.union(meta.added,all_changed,meta.deleted)meta.branch_tags={}branch_batches={}forbranch_pathinall_modifyed:branch_batches[branch_path]=[]#Building up the branches that have files on themforfinfiles_to_commit:subpath,branch_info=meta.split_path(f,r.revnum)ifnotbranch_info:continuebranch_path=branch_info[0]ifbranch_pathinall_modifyed:branch_batches[branch_path].append((subpath,f))# 1. handle tags commit first, because branches need tags informationforbranch_pathinsorted(branch_batches.keys()):''' TODO:add test case! Only when an tag is changed to commit. We don't commit either when it's added or deleted. NOTICE:When an tag is added with modification, we will commit it. '''ifbranch_pathintag_changed:files=branch_batches[branch_path]commit_branch(meta,branch_path,files,True)# 2. handle tags ancestors, getting those ancestors to updating .hgtags files.iftag_changes:branch_tags=meta.get_branch_tags(r.revnum,tag_deleted,set(tag_added+tag_changed))forbpinbranch_tags:tags_file=''.join(branch_tags[bp])meta.branch_tags[bp]=tags_fileifbpnotinbranch_batches:branch_batches[bp]=[]# 3. handle non-tags branch commit. they may be closed branchforbranch_pathinsorted(branch_batches.keys()):ifbranch_pathnotintag_changes:files=branch_batches[branch_path]commit_branch(meta,branch_path,files,False)

installing both hgsubversion AND hgsubversion.svnmeta helped, thanks. I'm running into more prolems now, however, although not directly with svnmeta (I think).

To reproduce, use Subvertpy bindings (I tried with v 0.7.4. and v 0.8.1) and try cloning the Terminals repository I linked to in my first post here. Here's the SVN link: https://Terminals.svn.codeplex.com/svn

Things get cloned fine until revision 20721, which has a "%20" in its name. The way this is handled is certainly a bug in SvnBridge that CodePlex uses. Specifically, this file: https://terminals.svn.codeplex.com/svn/!svn/bc/20721/Resources/Transport%20Examples.zip does not exist. Uri-encoding the '%' symbol doesnt help, as the "https://terminals.svn.codeplex.com/svn/!svn/bc/20721/Resources/Transport%2520Examples.zip" file does not exist either. However, this one does: "https://terminals.svn.codeplex.com/svn/!svn/bc/20721/Resources/Transport%252520Examples.zip". I've worked around this problem by checking for files with a '%' symbol in the name and downloading them via python code instead of using svn libs. A very rough hack, I know, but it got me through this file. Next, I encountered a problem at revisions 45315 and 45316. You see, there is a file named HidePanel.PNG that gets deleted in revision 45315, and added with a lower case extension (i.e. HidePanel.png) in revision 45316. SVN code in the extension *hgsubversion, subvertpy, OR svnmeta - I dont know* somehow bundles these two revisions into ONE, with the second change being listed without a file type, so the code breaks. My explanation sucks, so please just try cloning this repository to see what I mean. Problem happens with both Subvertpy and SWIG bindings.

I've tried the latest version, and it certainly seems better. I've moved away from Subvertpy to SWIG, however, since I couldn't get an x64 build of Subvertpy to compile, for whatever reason. I may try at a later point, but SWIG it is for me, for now at least.
With that said, I've noticed that you've only included codeplex changes to the subvertpy wrapper, so I did the same to the swig one. Things seem to be working fine now.

Edit: I've not updated to the very latest of svnmeta, and pulled in a few changes from the Terminals CodePlex repository. The first changest looked like it was pulled from an unrelated repo (see: http://www.twitpic.com/6zhfvm). I'm assuming this has something to do with the latest hash change detection, but not sure. I'll try a new clone of the repo now.

@dreamkxd
Sure. The repo is the Terminals one I mentioned above: https://terminals.svn.codeplex.com/svn
When cloning it using the very latest svnmeta, it starts cloning from r91726. I didn't wait long enough to see what happens, but this is clearly not the full history.

Notice that I also, instead of urllib.quote'ing the path for CodePlex, just replace the '%' with '%25' I think this should be enough for it to work correctly (and it does work correctly for me like this), but I may be wrong.

Okay, I'm going to go out on a limb here and mark this bug as »invalid«. Whatever the original issue was, it's been drowned out by discussion unrelated to this project. Please re-report the bug if you want us to look into it.