← or20_qwen27b_v2

rich_3180

failed NO_PATCH NO_PATCH UNSUBMITTED empty_patch(timeout) · 14 tool calls · 0 s · Textualize/rich

Task input

Fix double-width characters disappearing when wrapping

## Type of changes

- [x] Bug fix
- [ ] New feature
- [ ] Documentation / docstrings
- [ ] Tests
- [ ] Other

## Checklist

- [x] I've run the latest [black](https://github.com/psf/black) with default args on new code.
- [x] I've updated CHANGELOG.md and CONTRIBUTORS.md where appropriate.
- [x] I've added tests for new code.
- [x] I accept that @willmcgugan may be pedantic in the code review.

## Description

Update wrapping logic to fix issues with CJK charcters disappearing when the "fold" location sat *within* a double-width character. Ensure we retain browser logic of: 


> if there is no space on the current line, move to a new line, and if theres not enough space on the entire new line, fold the text over multiple lines at appropriate locations.

Adds some additional tests and docstrings, documentation etc.

Fixes #3176 

The wrapping process is overall still quite simple and doesn't match the browser in many cases. For example, wrapping does not consider punctuation (lines can begin with punctuation), and whitespace is handled differently (but practically speaking it seems sensible).

Tool calls (14)

#ToolArgumentsResult
1run_command
{
 "command": "ls /workspace/rich/ && grep -rn \"fold\" /workspace/rich/_wrap.py | head -30"
}
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "ls: /tmp/swe_work/or2
full
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "ls: /tmp/swe_work/or20_qwen27b_v2/rich_3180/a/workspace/rich/: No such file or directory\n", "exit_code": 1}}
2run_command
{
 "command": "cat /workspace/rich/_wrap.py"
}
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "cat: /tmp/swe_work/or
full
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "cat: /tmp/swe_work/or20_qwen27b_v2/rich_3180/a/workspace/rich/_wrap.py: No such file or directory\n", "exit_code": 1}}
3run_command
{
 "command": "pwd; ls /workspace 2>/dev/null | head; ls"
}
{"status": "ok", "stdout": "/private/workspace\nassets\nasv.conf.json\nasvhashfile\nbenchmarks\nCHANGELOG.md\nCODE_OF_CONDUCT.md\nCONTRIBUTING.md\nCONTRIBUTORS.
full
{"status": "ok", "stdout": "/private/workspace\nassets\nasv.conf.json\nasvhashfile\nbenchmarks\nCHANGELOG.md\nCODE_OF_CONDUCT.md\nCONTRIBUTING.md\nCONTRIBUTORS.md\ndocs\nexamples\nFAQ.md\nfaq.yml\nimgs\nLICENSE\nmake.bat\nMakefile\npoetry.lock\npyproject.toml\npytest.ini\nquestions\nREADME.cn.md\nREADME.de-ch.md\nREADME.de.md\nREADME.es.md\nREADME.fa.md\nREADME.fr.md\nREADME.hi.md\nREADME.id.md\nREADME.it.md\nREADME.ja.md\nREADME.kr.md\nREADME.md\nREADME.pl.md\nREADME.pt-br.md\nREADME.ru.md\nREADME.sv.md\nREADME.tr.md\nREADME.zh-tw.md\nrich\nSECURITY.md\nsetup.py\ntests\ntools\ntox.ini\n", "exit_code": 0}
4read_file
{
 "filepath": "rich/_wrap.py"
}
{"status": "ok", "filepath": "rich/_wrap.py", "content": "import re\nfrom typing import Iterable, List, Tuple\n\nfrom ._loop import loop_last\nfrom .cells impor
full
{"status": "ok", "filepath": "rich/_wrap.py", "content": "import re\nfrom typing import Iterable, List, Tuple\n\nfrom ._loop import loop_last\nfrom .cells import cell_len, chop_cells\n\nre_word = re.compile(r\"\\s*\\S+\\s*\")\n\n\ndef words(text: str) -> Iterable[Tuple[int, int, str]]:\n    position = 0\n    word_match = re_word.match(text, position)\n    while word_match is not None:\n        start, end = word_match.span()\n        word = word_match.group(0)\n        yield start, end, word\n        word_match = re_word.match(text, end)\n\n\ndef divide_line(text: str, width: int, fold: bool = True) -> List[int]:\n    divides: List[int] = []\n    append = divides.append\n    line_position = 0\n    _cell_len = cell_len\n    for start, _end, word in words(text):\n        word_length = _cell_len(word.rstrip())\n        if line_position + word_length > width:\n            if word_length > width:\n                if fold:\n                    chopped_words = chop_cells(word, max_size=width, position=0)\n                    for last, line in loop_last(chopped_words):\n                        if start:\n                            append(start)\n\n                        if last:\n                            line_position = _cell_len(line)\n                        else:\n                            start += len(line)\n                else:\n                    if start:\n                        append(start)\n                    line_position = _cell_len(word)\n            elif line_position and start:\n                append(start)\n                line_position = _cell_len(word)\n        else:\n            line_position += _cell_len(word)\n    return divides\n\n\nif __name__ == \"__main__\":  # pragma: no cover\n    from .console import Console\n\n    console = Console(width=10)\n    console.print(\"12345 abcdefghijklmnopqrstuvwyxzABCDEFGHIJKLMNOPQRSTUVWXYZ 12345\")\n    print(chop_cells(\"abcdefghijklmnopqrstuvwxyz\", 10, position=2))\n", "start_line": 1, "end_line": 56,
5run_command
{
 "command": "cd /workspace && git log --oneline -5 && git status | head -5"
}
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "/bin/bash: line 0: cd
full
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "/bin/bash: line 0: cd: /tmp/swe_work/or20_qwen27b_v2/rich_3180/a/workspace: No such file or directory\n", "exit_code": 1}}
6read_file
{
 "filepath": "rich/cells.py"
}
{"status": "ok", "filepath": "rich/cells.py", "content": "import re\nfrom functools import lru_cache\nfrom typing import Callable, List\n\nfrom ._cell_widths im
full
{"status": "ok", "filepath": "rich/cells.py", "content": "import re\nfrom functools import lru_cache\nfrom typing import Callable, List\n\nfrom ._cell_widths import CELL_WIDTHS\n\n# Regex to match sequence of the most common character ranges\n_is_single_cell_widths = re.compile(\"^[\\u0020-\\u006f\\u00a0\\u02ff\\u0370-\\u0482]*$\").match\n\n\n@lru_cache(4096)\ndef cached_cell_len(text: str) -> int:\n    \"\"\"Get the number of cells required to display text.\n\n    This method always caches, which may use up a lot of memory. It is recommended to use\n    `cell_len` over this method.\n\n    Args:\n        text (str): Text to display.\n\n    Returns:\n        int: Get the number of cells required to display text.\n    \"\"\"\n    _get_size = get_character_cell_size\n    total_size = sum(_get_size(character) for character in text)\n    return total_size\n\n\ndef cell_len(text: str, _cell_len: Callable[[str], int] = cached_cell_len) -> int:\n    \"\"\"Get the number of cells required to display text.\n\n    Args:\n        text (str): Text to display.\n\n    Returns:\n        int: Get the number of cells required to display text.\n    \"\"\"\n    if len(text) < 512:\n        return _cell_len(text)\n    _get_size = get_character_cell_size\n    total_size = sum(_get_size(character) for character in text)\n    return total_size\n\n\n@lru_cache(maxsize=4096)\ndef get_character_cell_size(character: str) -> int:\n    \"\"\"Get the cell size of a character.\n\n    Args:\n        character (str): A single character.\n\n    Returns:\n        int: Number of cells (0, 1 or 2) occupied by that character.\n    \"\"\"\n    return _get_codepoint_cell_size(ord(character))\n\n\n@lru_cache(maxsize=4096)\ndef _get_codepoint_cell_size(codepoint: int) -> int:\n    \"\"\"Get the cell size of a character.\n\n    Args:\n        codepoint (int): Codepoint of a character.\n\n    Returns:\n        int: Number of cells (0, 1 or 2) occupied by that character.\n    \"\"\"\n\n    _table = CELL_WIDTHS\
7run_command
{
 "command": "cd /private/workspace && git log --oneline -3 && head -30 CHANGELOG.md && python3 -c \"import rich; print(rich.__version__)\""
}
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "/bin/bash: line 0: cd
full
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "/bin/bash: line 0: cd: /private/workspace: No such file or directory\n", "exit_code": 1}}
8run_command
{
 "command": "pwd; ls -d /private/workspace 2>&1; ls -d /workspace 2>&1; echo $PWD"
}
{"status": "ok", "stdout": "/private/workspace\nls: /private/workspace: No such file or directory\nls: /tmp/swe_work/or20_qwen27b_v2/rich_3180/a/workspace: No s
full
{"status": "ok", "stdout": "/private/workspace\nls: /private/workspace: No such file or directory\nls: /tmp/swe_work/or20_qwen27b_v2/rich_3180/a/workspace: No such file or directory\n/private/workspace\n", "exit_code": 0}
9run_command
{
 "command": "git log --oneline -3; head -20 CHANGELOG.md; python3 -c \"import rich; print(rich.__version__, rich.__file__)\""
}
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "071fea5a baseline\n7b64494c Update
full
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "071fea5a baseline\n7b64494c Update pre-commit hooks (#3113)\n4adfca5a Fix #3104 (#3105)\n# Changelog\n\nAll notable changes to this project will be documented in this file.\n\nThe format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),\nand this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).\n\n## Unreleased\n\n### Added\n\n- Adds missing parameters to Panel.fit https://github.com/Textualize/rich/issues/3142\n\n### Fixed\n\n- Ensure font is correctly inherited in exported HTML https://github.com/Textualize/rich/issues/3104\n\n## [13.6.0] - 2023-09-30\n\n### Added\n", "stderr": "Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\n    import rich; print(rich.__version__, rich.__file__)\n                       ^^^^^^^^^^^^^^^^\nAttributeError: module 'rich' has no attribute '__version__'\n", "exit_code": 1}}
10run_command
{
 "command": "grep -rn \"divide_line\\|fold\" tests/test_text.py tests/test_wrap.py 2>/dev/null | head -30; ls tests | grep -i wrap"
}
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "", "exit_code": 1}}
full
{"status": "error", "error_type": "CommandError", "error_message": "Command failed with exit code 1", "details": {"stdout": "", "stderr": "", "exit_code": 1}}
11run_command
{
 "command": "cat > /tmp/repro.py <<'EOF'\nfrom rich._wrap import divide_line\nfrom rich.cells import cell_len\n\n# CJK text, no spaces, width 10\ntext = \"一二三四五六七八九十一二三四五六七八九十\"\ndivides = divide_line(text, 10)\nprint(\"divides:\", divides)\nlines = []\nprev = 0\nfor d in divides:\n    lines.append(text[prev:d])\n    prev = d\nlines.append(text[prev:])\nfor line in lines:\n    print(repr(line), cell_len(line))\nEOF\npython3 /tmp/repro.py"
}
{"status": "ok", "stdout": "divides: [5, 10, 15]\n'\u4e00\u4e8c\u4e09\u56db\u4e94' 10\n'\u516d\u4e03\u516b\u4e5d\u5341' 10\n'\u4e00\u4e8c\u4e09\u56db\u4e94' 10\
full
{"status": "ok", "stdout": "divides: [5, 10, 15]\n'\u4e00\u4e8c\u4e09\u56db\u4e94' 10\n'\u516d\u4e03\u516b\u4e5d\u5341' 10\n'\u4e00\u4e8c\u4e09\u56db\u4e94' 10\n'\u516d\u4e03\u516b\u4e5d\u5341' 10\n", "exit_code": 0}
12run_command
{
 "command": "cat > /tmp/repro2.py <<'EOF'\nfrom rich.cells import chop_cells, cell_len\nprint(chop_cells(\"一二三四五六七八九十\", 10))\nprint(chop_cells(\"一二三四五六七八九十\", 8))\nprint(chop_cells(\"abcdefghij\", 10))\nprint(chop_cells(\"abcdefghij\", 8))\nEOF\npython3 /tmp/repro2.py"
}
{"status": "ok", "stdout": "['\u5341\u4e5d\u516b\u4e03\u516d', '\u4e94\u56db\u4e09\u4e8c\u4e00']\n['\u5341\u4e5d\u516b\u4e03', '\u516d\u4e94\u56db\u4e09', '\u4e
full
{"status": "ok", "stdout": "['\u5341\u4e5d\u516b\u4e03\u516d', '\u4e94\u56db\u4e09\u4e8c\u4e00']\n['\u5341\u4e5d\u516b\u4e03', '\u516d\u4e94\u56db\u4e09', '\u4e8c\u4e00']\n['jihgfedcba']\n['jihgfedc', 'ba']\n", "exit_code": 0}
13run_command
{
 "command": "grep -n \"def wrap\" rich/text.py rich/console.py | head; grep -rn \"divide_line\" rich/ | head"
}
{"status": "ok", "stdout": "rich/text.py:1198: def wrap(\nrich/_wrap.py:20:def divide_line(text: str, width: int, fold: bool = True) -> List[int]:\nrich/text.py
full
{"status": "ok", "stdout": "rich/text.py:1198:    def wrap(\nrich/_wrap.py:20:def divide_line(text: str, width: int, fold: bool = True) -> List[int]:\nrich/text.py:20:from ._wrap import divide_line\nrich/text.py:1233:                offsets = divide_line(str(line), width, fold=wrap_overflow == \"fold\")\n", "exit_code": 0}
14read_file
{
 "filepath": "rich/text.py",
 "start_line": 1198,
 "end_line": 1260
}
{"status": "ok", "filepath": "rich/text.py", "content": " def wrap(\n self,\n console: \"Console\",\n width: int,\n *,\n justify: Optional[\"JustifyMethod\"] =
full
{"status": "ok", "filepath": "rich/text.py", "content": "    def wrap(\n        self,\n        console: \"Console\",\n        width: int,\n        *,\n        justify: Optional[\"JustifyMethod\"] = None,\n        overflow: Optional[\"OverflowMethod\"] = None,\n        tab_size: int = 8,\n        no_wrap: Optional[bool] = None,\n    ) -> Lines:\n        \"\"\"Word wrap the text.\n\n        Args:\n            console (Console): Console instance.\n            width (int): Number of cells available per line.\n            justify (str, optional): Justify method: \"default\", \"left\", \"center\", \"full\", \"right\". Defaults to \"default\".\n            overflow (str, optional): Overflow method: \"crop\", \"fold\", or \"ellipsis\". Defaults to None.\n            tab_size (int, optional): Default tab size. Defaults to 8.\n            no_wrap (bool, optional): Disable wrapping, Defaults to False.\n\n        Returns:\n            Lines: Number of lines.\n        \"\"\"\n        wrap_justify = justify or self.justify or DEFAULT_JUSTIFY\n        wrap_overflow = overflow or self.overflow or DEFAULT_OVERFLOW\n\n        no_wrap = pick_bool(no_wrap, self.no_wrap, False) or overflow == \"ignore\"\n\n        lines = Lines()\n        for line in self.split(allow_blank=True):\n            if \"\\t\" in line:\n                line.expand_tabs(tab_size)\n            if no_wrap:\n                new_lines = Lines([line])\n            else:\n                offsets = divide_line(str(line), width, fold=wrap_overflow == \"fold\")\n                new_lines = line.divide(offsets)\n            for line in new_lines:\n                line.rstrip_end(width)\n            if wrap_justify:\n                new_lines.justify(\n                    console, width, justify=wrap_justify, overflow=wrap_overflow\n                )\n            for line in new_lines:\n                line.truncate(width, overflow=wrap_overflow)\n            lines.extend(new_lines)\n        return lines\n\n    def fit(se