Suddenly no gpu deteced, ratelimit, and no msa output... #252
Comments
|
Having the same issue. |
|
I also have the same issue. HTTPSConnectionPool(host='api.colabfold.com', port=443): Read timed out. |
|
Probably this issue is related to these threads: I hope the server issue will be fixed soon. |
|
Please check if the issue is still happening at everyone who had issues. Our IT claims to have resolved the issue, it might take a while to propagate though. |
As for me, the issue has now been resolved. Thanks for your magic🪄. |
|
Thanks a lot! I’d appreciate feedback from more people in this thread if this is resolved, as it didn’t seem very deterministic when this happens. |
|
I'm still seeing this error (Read timed out) as of just now. It cropped up yesterday afternoon sometime |
|
@Drew-Thomson could you please run: |
|
Thanks for looking at this! I get: (base) drew@obelisk:~$ tracepath api.colabfold.com
|
|
@milot-mirdita I unfortunately am also still having issues - my tracepath output is essentially the same as @Drew-Thomson above but with no intermediate paths or replies (all no reply):
|
|
I have implemented a workaround until our IT can fully solve this. Afterwards, colabfold's MSA server should work in a reasonable speed again |
|
Can confirm that ping gives 205.185.124.98. I've just started a colabfold job and it has run the MSA and is producing models. Thank you so much for you help here- very much appreciated! |
|
@milot-mirdita Thank you so much for all of the work! Everything seems to be working in terms of the MSA now. To confirm, when I run the command, I get the following (indicating that the new IP has replaced the old in DNS):
When I run colabfold using the provided Docker container, it seems to get past the stage of querying the API for MSA. Thank you so much! |
Hello local colabolfd team,
I've been using local colabfold for a while. However, suddenly since few hours ago I've been having problems such as no gpu deteced, ratelimit, and no msa output... I tried reinstalling colabfold but it didn't solve the problem...My gpu runs fine on other tools so I do not think it is problem with the gpu...
2024-08-27 13:17:29,133 Running colabfold 1.5.5 (fdf3b235b88746681c46ea12bcded76ecf8e1f76)
2024-08-27 13:17:29,198 WARNING: no GPU detected, will be using CPU
2024-08-27 13:17:30,144 Found 8 citations for tools or databases
2024-08-27 13:17:30,144 Query 1/1: fl_rd_fl_rd_2-7_1_0_3fixed.fa1 (length 306)
2024-08-27 13:17:47,379 Could not get MSA/templates for fl_rd_fl_rd_2-7_1_0_3fixed.fa1: HTTPSConnectionPool(host='api.colabfold.com', port=443): Read timed out.
Traceback (most recent call last):
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/urllib3/response.py", line 748, in _error_catcher
yield
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/urllib3/response.py", line 873, in _raw_read
data = self._fp_read(amt, read1=read1) if not fp_closed else b""
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/urllib3/response.py", line 856, in _fp_read
return self._fp.read(amt) if amt is not None else self._fp.read()
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/http/client.py", line 460, in read
return self._read_chunked(amt)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/http/client.py", line 592, in _read_chunked
value.append(self._safe_read(chunk_left))
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/http/client.py", line 631, in _safe_read
data = self.fp.read(amt)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/socket.py", line 705, in readinto
return self._sock.recv_into(b)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/ssl.py", line 1307, in recv_into
return self.read(nbytes, buffer)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/ssl.py", line 1163, in read
return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/colabfold/batch.py", line 1467, in run
= get_msa_and_templates(jobname, query_sequence, a3m_lines, result_dir, msa_mode, use_templates,
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/colabfold/batch.py", line 778, in get_msa_and_templates
a3m_lines_mmseqs2, template_paths = run_mmseqs2(
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/colabfold/colabfold.py", line 295, in run_mmseqs2
tar.extractall(path=TMPL_PATH)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2264, in extractall
self._extract_one(tarinfo, path, set_attrs=not tarinfo.isdir(),
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2327, in _extract_one
self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2410, in _extract_member
self.makefile(tarinfo, targetpath)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 2463, in makefile
copyfileobj(source, target, tarinfo.size, ReadError, bufsize)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 252, in copyfileobj
buf = src.read(bufsize)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 526, in read
buf = self._read(size)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/tarfile.py", line 544, in _read
buf = self.fileobj.read(self.bufsize)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/urllib3/response.py", line 949, in read
data = self._raw_read(amt)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/urllib3/response.py", line 872, in _raw_read
with self._error_catcher():
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/contextlib.py", line 153, in exit
self.gen.throw(typ, value, traceback)
File "/home/jehoon/colabfold2/localcolabfold/colabfold-conda/lib/python3.10/site-packages/urllib3/response.py", line 753, in _error_catcher
raise ReadTimeoutError(self._pool, None, "Read timed out.") from e # type: ignore[arg-type]
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.colabfold.com', port=443): Read timed out.
2024-08-27 13:17:47,394 Done
The text was updated successfully, but these errors were encountered: