>> # Create a DataSource, use os.curdir (default) for local storage. >>> from numpy import DataSource >>> ds = DataSource() >>> >>> # Open a remote file. >>> # DataSource downloads the file, stores it locally in: >>> # './www.google.com/index.html' >>> # opens the file and returns a file object. >>> fp = ds.open('http://www.google.com/') # doctest: +SKIP >>> >>> # Use the file as you normally would >>> fp.read() # doctest: +SKIP >>> fp.close() # doctest: +SKIP N) set_modulectSU;aSU;a[SU<35egUb [S5eUb [S5eg)zCheck mode and that encoding and newline are compatible. Parameters ---------- mode : str File open mode. encoding : str File encoding. newline : str Newline for text files. tbzInvalid mode: Nz0Argument 'encoding' not supported in binary modez/Argument 'newline' not supported in binary mode) ValueErrormodeencodingnewlines ځ/app/backend/data/users/69c26b92-e8f0-4417-86e0-3a8ede0c9e85/documents/venv/lib/python3.13/site-packages/numpy/lib/_datasource.py _check_moder ,sT d{ $;~dX67 7   OP P  NO O c0\rSrSrSrSrSrSrSrSr g) _FileOpenersHa Container for different methods to open (un-)compressed files. `_FileOpeners` contains a dictionary that holds one method for each supported file format. Attribute lookup is implemented in such a way that an instance of `_FileOpeners` itself can be indexed with the keys of that dictionary. Currently uncompressed files as well as files compressed with ``gzip``, ``bz2`` or ``xz`` compression are supported. Notes ----- `_file_openers`, an instance of `_FileOpeners`, is made available for use in the `_datasource` module. Examples -------- >>> import gzip >>> np.lib._datasource._file_openers.keys() [None, '.bz2', '.gz', '.xz', '.lzma'] >>> np.lib._datasource._file_openers['.gz'] is gzip.open True c,SUlS[0Ulg)NF)_loadedopen _file_openersselfs r __init___FileOpeners.__init__as "D\rcUR(agSSKnURURS'SSKnURURS'SSKnURURS'URURS'SUlg![a Nif=f![a N[f=f![[4a N:f=f)Nrz.bz2z.gzz.xzz.lzmaT)rbz2rr ImportErrorgziplzmaAttributeError)rrrrs r _load_FileOpeners._loades <<   ),D  v &  (, D  u %  (, D  u %*.))D  w '  %       ^,   s4BB6B. BB B+*B+.CCchUR5 [URR55$)a Return the keys of currently supported file openers. Parameters ---------- None Returns ------- keys : list The keys are None for uncompressed files and the file extension strings (i.e. ``'.gz'``, ``'.xz'``) for supported compression methods. )r listrkeysrs r r$_FileOpeners.keyss' D&&++-..rc@UR5 URU$N)r r)rkeys r __getitem___FileOpeners.__getitem__s !!#&&r)rrN) __name__ __module__ __qualname____firstlineno____doc__rr r$r)__static_attributes__rr rrHs0*6/&'rrrc8[U5nURXX4S9$)a Open `path` with `mode` and return the file object. If ``path`` is an URL, it will be downloaded, stored in the `DataSource` `destpath` directory and opened from there. Parameters ---------- path : str or pathlib.Path Local file path or URL to open. mode : str, optional Mode to open `path`. Mode 'r' for reading, 'w' for writing, 'a' to append. Available modes depend on the type of object specified by path. Default is 'r'. destpath : str, optional Path to the directory where the source file gets downloaded to for use. If `destpath` is None, a temporary directory will be created. The default path is the current directory. encoding : {None, str}, optional Open text file with given encoding. The default encoding will be what `open` uses. newline : {None, str}, optional Newline to use when reading text file. Returns ------- out : file object The opened file. Notes ----- This is a convenience function that instantiates a `DataSource` and returns the file object from ``DataSource.open(path)``. r r ) DataSourcer)pathr destpathr r dss r rrs"J H B 7747 BBrznumpy.lib.npyioc\rSrSrSr\R 4SjrSrSr Sr Sr Sr S r S rS rS rS rSrSSjrSrg)r5a DataSource(destpath='.') A generic data source file (file, http, ftp, ...). DataSources can be local files or remote files/URLs. The files may also be compressed or uncompressed. DataSource hides some of the low-level details of downloading the file, allowing you to simply pass in a valid file path (or URL) and obtain a file object. Parameters ---------- destpath : str or None, optional Path to the directory where the source file gets downloaded to for use. If `destpath` is None, a temporary directory will be created. The default path is the current directory. Notes ----- URLs require a scheme string (``http://``) to be used, without it they will fail:: >>> repos = np.lib.npyio.DataSource() >>> repos.exists('www.google.com/index.html') False >>> repos.exists('http://www.google.com/index.html') True Temporary directories are deleted when the DataSource is deleted. Examples -------- :: >>> ds = np.lib.npyio.DataSource('/home/guido') >>> urlname = 'http://www.google.com/' >>> gfile = ds.open('http://www.google.com/') >>> ds.abspath(urlname) '/home/guido/www.google.com/index.html' >>> ds = np.lib.npyio.DataSource(None) # use with temporary file >>> ds.open('/home/guido/foobar.txt') >>> ds.abspath('/home/guido/foobar.txt') '/tmp/.../home/guido/foobar.txt' cU(a,[RRU5UlSUlgSSKnUR 5UlSUlg)z2Create a DataSource with a local path at destpath.FrNT)osr6abspath _destpath _istmpdesttempfilemkdtemp)rr7r@s r rDataSource.__init__s: WW__X6DN#DO %--/DN"DOrc[US5(a2UR(a SSKnURUR5 ggg)Nr?r)hasattrr?shutilrmtreer>)rrEs r __del__DataSource.__del__s0 4 & &4??  MM$.. ),; &rcr[RRU5up#U[R 5;$)zNTest if the filename is a zip file by looking at the file extension. )r<r6splitextrr$)rfilenamefnameexts r _iszipDataSource._iszips.WW%%h/ m((***rc2^Sm[U4SjU55$)z4Test if the given mode will open a file for writing.)w+c3,># UH oT;v M g7fr'r1).0c _writemodess r *DataSource._iswritemode..s2T #Ts)any)rr rVs @r _iswritemodeDataSource._iswritemode s! 2T222rctURU5(a[RRU5$US4$)zaSplit zip extension from filename and return filename. Returns ------- base, zip_ext : {tuple} N)rNr<r6rJ)rrKs r _splitzipextDataSource._splitzipexts2 ;;x 77##H- -T> !rcU/nURU5(d7[R5HnU(dM URX-5 M! U$)z9Return a tuple containing compressed filename variations.)rNrr$append)rrKnameszipexts r _possible_namesDataSource._possible_names"sF {{8$$',,.6LL!23/ rcNSSKJn U"U5up4pVpx[U=(a U5$)z=Test if path is a net location. Tests the scheme and netloc.rurlparse) urllib.parsergbool) rr6rgschemenetlocupathuparamsuqueryufrags r _isurlDataSource._isurl+s* *9A5F%v&&rc*SSKnSSKJn URU5n[R R [R RU55(d3[R"[R RU55 URU5(a9U"U5n[US5nURXV5 SSS5 SSS5 U$URX5 U$!,(df  N+=f!,(df  U$=f)zXCache the file specified by path. Creates a copy of the file in the datasource cache. rNurlopenwb) rEurllib.requestrtr=r<r6existsdirnamemakedirsrp_open copyfileobjcopyfile)rr6rErtrl openedurlfs r _cacheDataSource._cache:s * T"ww~~bggooe455 KK. / ;;t  )5$'1&&y4(   OOD ( ('  s$. D;C2 D2 D <D DcURU5(d4URU5nX RURU55- nO3URURU55nX RU5-nUHDnURU5(dMURU5(aUR U5nUs $ g)aISearches for ``path`` and returns full path if found. If path is an URL, _findfile will cache a local copy and return the path to the cached file. If path is a local file, _findfile will return a path to that local file. The search will include possible compressed versions of the file and return the first occurrence found. N)rprcr=rwr)rr6filelistnames r _findfileDataSource._findfileTs{{4  ++D1H ,,T\\$-?@ @H++DLL,>?H"6"6t"<len_sanitize_relative_pathr<r6join) rr6rg splitpathrjrkrlrmrnros r r=DataSource.abspathss{0 *JJt~~q1 y>A Q>$   ** T" 77>>%  ;;t   !$-   s?BB#"B#Nc.URU5(a!URU5(a [S5eURU5nU(a9UR U5upgUS:XaUR SS5 [ U"XRX4S9$[US35e)a Open and return file-like object. If `path` is an URL, it will be downloaded, stored in the `DataSource` directory and opened from there. Parameters ---------- path : str or pathlib.Path Local file path or URL to open. mode : {'r', 'w', 'a'}, optional Mode to open `path`. Mode 'r' for reading, 'w' for writing, 'a' to append. Available modes depend on the type of object specified by `path`. Default is 'r'. encoding : {None, str}, optional Open text file with given encoding. The default encoding will be what `open` uses. newline : {None, str}, optional Newline to use when reading text file. Returns ------- out : file object File object. zURLs are not writeablerrRrz not found.)rprZrrr]replacerFileNotFoundError)rr6r r r found_fnamerMs r rDataSource.opensD ;;t  !2!24!8!856 6t$ ++E2KFe| S"% %e/7J J$tfK$89 9r)r>r?r2NN)r+r,r-r.r/r<curdirrrGrNrZr]rcrprrr=rrwrr0r1rr r5r5sT.`!# #*+3 " '4>(;T 6p.:rr5cf\rSrSrSr\R 4SjrSrSr Sr Sr Sr S S jr S rS rg ) Repositoryia Repository(baseurl, destpath='.') A data repository where multiple DataSource's share a base URL/directory. `Repository` extends `DataSource` by prepending a base URL (or directory) to all the files it handles. Use `Repository` when you will be working with multiple files from one base URL. Initialize `Repository` with the base URL, then refer to each file by its filename only. Parameters ---------- baseurl : str Path to the local directory or remote location that contains the data files. destpath : str or None, optional Path to the directory where the source file gets downloaded to for use. If `destpath` is None, a temporary directory will be created. The default path is the current directory. Examples -------- To analyze all files in the repository, do something like this (note: this is not self-contained code):: >>> repos = np.lib._datasource.Repository('/home/user/data/dir/') >>> for filename in filelist: ... fp = repos.open(filename) ... fp.analyze() ... fp.close() Similarly you could use a URL for a repository:: >>> repos = np.lib._datasource.Repository('http://www.xyz.edu/data') c6[RXS9 Xlg)z>Create a Repository with a shared url or directory of baseurl.)r7N)r5r_baseurl)rbaseurlr7s r rRepository.__init__<sD4 rc.[RU5 gr')r5rGrs r rGRepository.__del__As4 rcURURS5n[U5S:Xa,[RR URU5nU$UnU$)z>Return complete path for path. Prepends baseurl if necessary.rr)rrrr<r6r)rr6rresults r _fullpathRepository._fullpathDsMJJt}}a0 y>Q WW\\$--6F F rcJ[RXRU55$)z8Extend DataSource method to prepend baseurl to ``path``.)r5rrrr6s r rRepository._findfileMs##D..*>??rcJ[RXRU55$)a Return absolute path of file in the Repository directory. If `path` is an URL, then `abspath` will return either the location the file exists locally or the location it would exist when opened using the `open` method. Parameters ---------- path : str or pathlib.Path Can be a local file or a remote URL. This may, but does not have to, include the `baseurl` with which the `Repository` was initialized. Returns ------- out : str Complete path, including the `DataSource` destination directory. )r5r=rrs r r=Repository.abspathQs*!!$t(<==rcJ[RXRU55$)a% Test if path exists prepending Repository base URL to path. Test if `path` exists as (and in this order): - a local file. - a remote URL that has been downloaded and stored locally in the `DataSource` directory. - a remote URL that has not been downloaded, but is valid and accessible. Parameters ---------- path : str or pathlib.Path Can be a local file or a remote URL. This may, but does not have to, include the `baseurl` with which the `Repository` was initialized. Returns ------- out : bool True if `path` exists. Notes ----- When `path` is an URL, `exists` will return True if it's either stored locally in the `DataSource` directory, or is a valid remote URL. `DataSource` does not discriminate between the two, the file is accessible if it exists in either location. )r5rwrrs r rwRepository.existshs@  ~~d';< ;;t}} % %%AC C::dmm, ,r)rr)r+r,r-r.r/r<rrrGrrr=rwrrr0r1rr rrs=%N*, !@>. =DC@-rr) r/r< numpy._utilsrrrzr rrrr5rr1rr rsy#H # P8M'M'` "))dD&CR M:M:M:` h-*h-r