{"id":232,"date":"2008-03-03T10:15:08","date_gmt":"2008-03-03T17:15:08","guid":{"rendered":"https:\/\/dubinko.info\/blog\/2008\/03\/03\/webpath-and-wikipedia\/"},"modified":"2008-03-03T23:38:44","modified_gmt":"2008-03-04T06:38:44","slug":"webpath-and-wikipedia","status":"publish","type":"post","link":"https:\/\/dubinko.info\/blog\/2008\/03\/webpath-and-wikipedia\/","title":{"rendered":"WebPath and Wikipedia"},"content":{"rendered":"<p>The <a href=\"http:\/\/sourceforge.net\/projects\/webpath\">WebPath<\/a> bug reports continue to roll in. For one, queries against *.wikipedia.* don&#8217;t seem to work. You get <em>something<\/em> back, but it has no resemblance to the page you were looking for. The problem comes from the <a href=\"http:\/\/cgi.w3.org\/cgi-bin\/tidy\">W3C tidy service<\/a> that I use, specifically that the (understandably overworked and understaffed) admins at the Wikimedia Foundation seem to have blocked it. It seems like more than a simple IP or user-agent-based block. I&#8217;ve emailed them about it but haven&#8217;t heard back yet.<\/p>\n<p>So, this highlights the limitation of having a single-source converter in the <a href=\"http:\/\/webpath.svn.sourceforge.net\/viewvc\/webpath\/trunk\/platonicweb.py?revision=10&amp;view=markup\">Platonic Web<\/a> module of WebPath. So I turn to my readers: do you know of any other tidy servers? Or converters of a non-tidy origin? For any of these to work, they need to return clean XML corresponding to the original page (as opposed to, say, returning something with big headers\/footers or ampersand-encoded). This seems like an outstanding need for the open source community.<\/p>\n<p>Please comment below with ideas. Thanks! -m<\/p>\n<p>UPDATE: heard back from the Wikipedia admins, and although professional and helpful-as-can-be-expected, they won&#8217;t be changing anything on their end. Still looking for more open source options.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The WebPath bug reports continue to roll in. For one, queries against *.wikipedia.* don&#8217;t seem to work. You get something back, but it has no resemblance to the page you were looking for. The problem comes from the W3C tidy service that I use, specifically that the (understandably overworked and understaffed) admins at the Wikimedia&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[33,6,22,32],"tags":[],"class_list":["post-232","post","type-post","status-publish","format-standard","hentry","category-annoyance","category-intentional-web","category-software","category-xpath"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8eo8l-3K","_links":{"self":[{"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/posts\/232","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/comments?post=232"}],"version-history":[{"count":0,"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/posts\/232\/revisions"}],"wp:attachment":[{"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/media?parent=232"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/categories?post=232"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dubinko.info\/blog\/wp-json\/wp\/v2\/tags?post=232"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}