patch from qres

Faré <fahree <at> gmail.com>
2013-09-20 23:41:57 GMT

I have a patch for babel as used by qres.
It was originally written by Scott McKay and Dan Weinreb (RIP), to
1- add support for CESU-8 (as infamously used by Oracle).
2- speed up babel (and measure the speed)
Plus by me to
3- support XCVB.
I've removed some extraneous type declarations from SWM&DLW,
where some other macro already declared a type, and also removed
some handling for surrogate codes that was already added upstream since.
The patch is attached (against HEAD on github).
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
Invent a clever saying, and your name shall live forever.
— Anonymous

Introspection

Frank <fau <at> riseup.net>
2013-08-19 08:17:17 GMT

Hello,
I would like to determine the (max) number of bytes a character/units in
a certain encoding consumes. Unfortunately there does not seem to be an
interface exposed for it (or I just don't see it).
I'm doing it this way:
(defun max-code-units (encoding)
(babel::enc-max-units-per-char (babel::get-character-encoding
encoding)))
(defun code-unit-size (encoding)
(ash (babel::enc-code-unit-size (babel::get-character-encoding
encoding)) -3))
(defun max-code-size (encoding)
(let ((e (babel::get-character-encoding encoding)))
(* (babel::enc-max-units-per-char e)
(ash (babel::enc-code-unit-size e) -3))))
Is this a good idea to to that way? Please comment.
Thanks,
Frank

BABEL-TESTS on ABCL

Today we've been working on ABCL being able to compile and load BABEL; actually, it could do that already, but didn't work with BABEL-TESTS. We've committed some fixes which handle reader issues uncovered by the library, but we're now stuck: there's a bit of code commented out on CCL. It seems we're running into the same problems as CCL, due to strictness of interpretation of code points.

Could you mark the #-ccl section in tests/tests.lisp as #-(or ccl abcl)? That solves our problem with the #\udcf0 code point.

error downloading babel

<harven <at> free.fr>
2012-03-23 15:35:09 GMT

Hello,
I encounter a problem trying to download babel using quicklisp.
I am on debian squeeze and I use the last version of sbcl 1.0.55 that I
downloaded 5mns ago from the website, together with a fresh install of
quicklisp. I can reproduce the error using the version of sbcl 1.0.40
in the debian repositories.
The relevant part of the log which is attached to this message seems to be
* (ql:quickload "babel")
To load "babel":
Load 1 ASDF system:
babel
; Loading "babel"
; file: /home/harven/.quicklisp/dists/quicklisp/software/babel-20120208-git/src/enc-unicode.lisp
; in: DEFINE-UTF-16 :UTF-16
; (BABEL-ENCODINGS::DEFINE-UTF-16 :UTF-16)
;
; caught ERROR:
; (during macroexpansion of (DEFINE-UTF-16 :UTF-16))
; #:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
; Wanted one of (STRING SIMPLE-STRING).
A similar error has been reported on github
https://github.com/cl-babel/babel/issues/9
where it was suggested to run the command
(alexandria:format-symbol t '#:~a-code-point-counter "UTF16")
Here is what I get.
* (ql:quickload "alexandria")
To load "alexandria":
Load 1 ASDF system:
alexandria
; Loading "alexandria"
("alexandria")
* (alexandria:format-symbol t '#:~a-code-point-counter "UTF16")
debugger invoked on a SB-KERNEL:CASE-FAILURE in thread
#<THREAD "initial thread" RUNNING {AB2C889}>:
#:~A-CODE-POINT-COUNTER fell through ETYPECASE expression.
Wanted one of (STRING SIMPLE-STRING).
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(SB-KERNEL:CASE-FAILURE
ETYPECASE
#:~A-CODE-POINT-COUNTER
(STRING SIMPLE-STRING))
0] 0
Any help ?

Problem with gbk-map.lisp

Luís Oliveira <luismbo <at> gmail.com>
2012-01-02 22:05:53 GMT

Hello Wenpeng,
I missed an obvious problem with your patch, it depends on Lisps
loading gbk-map.lisp using the UTF-8 encoding. AFAIK, there's no good
way to portably enforce that via ASDF, so I've converted the file to
an ASCII representation. It's not as pretty as the previous version,
but it seems to work.
Cheers,
--
--
Luís Oliveira
http://r42.eu/~luis/

Unit tests failures on different lisps

Anton Vodonosov <avodonosov <at> yandex.ru>
2011-12-28 21:21:34 GMT

Hello.
I am running tests of some most often Quicklisp-downloaded libraries, including babel.
Babel tests have different number of failures/errors on different Lisps (about 8, 9 or 5).
You may find the results here: http://common-lisp.net/project/cl-test-grid/pivot_ql-lib_lisp.html
Clicking the ok/fail status refer to the library test logs where you may find what failures
occurred.
Best regards,
- Anton

Re: Submit a GBK patch

Xiaofeng Yang <n.akr.akiiya <at> gmail.com>
2011-12-22 14:55:37 GMT

I suggest this for both GB2312 and GBK encoding (this 2 encodings are very important for Chinese users):http://trac.clozure.com/ccl/changeset/14911
I've tested this for CCL using the encoding tables and tests from GNU's iconv.

Most babel code came from CCL. I think to integrated this for babel is easily and more suitable though it just only for CCL now.

Days ago, I found this project babel, and tried to use it in my project, as I'm Chinese,I found it doesn't support GBK encoding in babel, so I wrote a patch for babel to makeit support GBK.

I just used common lisp for not a long time, so if there was something notgood enough, please let me know, I also hope you can accept this patch or help me modify it to make babel support GBK, so we can use it freely to process Chinese text.

Submit a GBK patch

Days ago, I found this project babel, and tried to use it in my project, as I'm Chinese,I found it doesn't support GBK encoding in babel, so I wrote a patch for babel to makeit support GBK.

I just used common lisp for not a long time, so if there was something notgood enough, please let me know, I also hope you can accept this patch or help me modify it to make babel support GBK, so we can use it freely to process Chinese text.