o Gfht4@sdZddlZddlZddlZddlmZmZmZmZddl Z ddl m Z ddl m Z e e dZe ddZe d Ze d d Ze d d ZddlmZe ddZeeZGdddZdedefddZdS)z Document Analyzer Module This module uses Claude LLM to analyze extracted document text and identify document types, extract key information, and provide descriptions. N)DictAnyListOptional)datetime) load_dotenvANTHROPIC_API_KEYANTHROPIC_MODELzclaude-3-5-haiku-latestOPENAI_API_KEY OPENAI_MODELz Qwen/Qwen3-Next-80B-A3B-Instruct OPENAI_URLz#https://api.deepinfra.com/v1/openai)OpenAI GENAI_ENGINEclaudec@seZdZdZdefddZdefddZded edeeeffd d Z d eeefdeeeeefffd dZ deeeeeffdeeeffddZ dS)DocumentAnalyzerzp Document analyzer that uses Claude to analyze document content and extract structured information. claude_api_keycCsdtdkr tj|d|_n tdkrtttd|_gdgdgdgdgd gd gd d |_d S)z Initialize the document analyzer. Args: claude_api_key (str): API key for Claude r)api_keyopen_llm)rbase_url)Incorporation_CertificateMOAAOAPAN_CardPAN_CertificateGST_CertificateMSME_CertificateMSME_RegistrationCIN_Certificate Director_PANDirector_Aadhaar Address_ProofLease_AgreementPartnership_DeedLicense Trade_LicenseRegistration_Certificate) ITRIncome_Tax_Return Balance_Sheet Audit_ReportCA_CertificateTurnover_CertificateRevenue_CertificateFinancial_StatementBank_StatementTax_Compliance_CertificateCopy_of_Cheque)ISO_CertificateCMMI_CertificatePatent_CertificateMembership_CertificateQuality_CertificateCompliance_CertificateAward_CertificateTraining_Certificate) Work_OrderPurchase_OrderContract AgreementCompletion_CertificatePerformance_CertificateClient_CertificateProject_CertificateReference_Letter Testimonial) Product_BrochureTechnical_Specification Price_List Quotation OEM_LetterAuthorization_LetterProduct_CertificateTechnical_DocumentManual Datasheet)Employee_ResumeEmployee_CertificateSalary_CertificateExperience_LetterAppointment_Letter Employee_ListEmployee_Details)LetterEmailCorrespondenceInvoiceReceiptGeneral_Document Miscellaneous)Legal Financial Certificates Experience TechnicalHROtherN) llm_model anthropic Anthropic claude_clientr r r openaidocument_categories)selfrribsz8DocumentAnalyzer.get_analysis_prompt..z You are an expert document analyzer. Analyze the provided document text and extract structured information. DOCUMENT CATEGORIES AND TYPES: a Your task is to: 1. Identify the document type using the format "Category:Subtype" (e.g., "Financial:ITR", "Experience:Work_Order") 2. Extract key information relevant to the document type 3. Provide a concise description 4. Note the original filename if mentioned RESPONSE FORMAT - Return ONLY a valid JSON object with this exact structure: { "doc_type": "Category:Subtype", "key_info": { "key1": "value1", "key2": "value2" }, "description": "Brief description of the document and its contents", "confidence": 0.95 } KEY INFORMATION EXTRACTION GUIDELINES: - For Financial documents (ITR, Balance Sheet, etc.): Extract assessment year, amounts, company name, total revenue - For Experience documents (Work Orders, Contracts, etc.): Extract customer, location, project details, value, dates - For Legal documents: Extract company name, registration numbers, dates, document number/ID - For Certificates: Extract issuing authority, validity period, certificate number, scope, purpose of certificate - For Technical documents: Extract product details, specifications, prices, suppliers, OEM name IMPORTANT RULES: - Always respond with valid JSON only - Use "Other:General_Document" if the document type cannot be clearly determined - Extract dates in a consistent format when possible - Include currency symbols and units where applicable - Set confidence between 0.1 and 1.0 based on how certain you are about the classification - If key information is not available, use empty strings or null values - Keep descriptions concise but informative (1-3 sentences) - If the document is signed or verified or attested, then you may please ignore the information about signing authority or person DO NOT include any text outside of the JSON structure. )rorgitems)rhcategories_textririrjget_analysis_prompt[s z$DocumentAnalyzer.get_analysis_prompt document_text file_pathc Cs4z|}tj|}d|d|}t|dkr!|ddd}tdkrC|jjjt ddd d |d |d gd }|j dj }n"tdkre|j jjjtdd d d |d |d gd}|jdjj }|drv|dddd }z2t|}||d<||d<t|d<gd} | D] } | |vrd|| <qd|vrd|d<|WWStjy} z,td|d| td|did|d||td d!WYd} ~ WSd} ~ wwty} z+td"|d| did#t| d|tj|tt| d!WYd} ~ Sd} ~ ww)$a* Analyze a single document and extract structured information. Args: document_text (str): Extracted text from the document file_path (str): Path to the original file Returns: Dict[str, Any]: Analysis results z FILENAME: z DOCUMENT CONTENT: iNz# [Document truncated due to length]rig?zYou are an expert document analyzer. Analyze documents and return structured JSON data about their type, key information, and description.userz ANALYZE THIS DOCUMENT: )rolecontent)model max_tokens temperaturesystemmessagesrrr)r|rz```jsonz```rx file_name analysis_date)doc_typekey_info description confidenceg?z"Failed to parse JSON response for rnzResponse was: zOther:General_Documentz8Document analysis failed - could not parse response for zJSON parsing failed)rrrrrxrrerrorzError analyzing document z'Document analysis failed due to error: )rvospathbasenamelenrbrercreater r{textstriprfchat completionsr choicesmessage startswithreplacejsonloadsrnow isoformatJSONDecodeErrorloggerr Exceptionstr) rhrwrxpromptfilename full_textresponse response_textanalysis_resultrequired_fieldsfielderirirjanalyze_documents           z!DocumentAnalyzer.analyze_documentdocuments_textc Csi}t|}td|dt|dD];\}\}}td|d|dtj||||}|||<| dd}| d d } td |d | d dqtd|d|S)a! Analyze multiple documents in batch. Args: documents_text (Dict[str, str]): Dictionary mapping file paths to extracted text Returns: Dict[str, Dict[str, Any]]: Dictionary mapping file paths to analysis results zStarting analysis of z documentszAnalyzing document /rnrUnknownrrz -> Classified as: z (confidence: z.2f)zCompleted analysis of ) rrinfo enumeratertrrrrget) rhranalysis_results total_docsirxrrrrririrjanalyze_documents_batchs $   z(DocumentAnalyzer.analyze_documents_batchrcCs|sdiddSi}g}|D]$}|dd}|dd}||vr+||d7<nd||<||q|r?t|t|nd}t|||dd|Dd S) z Generate a summary of the analysis results. Args: analysis_results (Dict[str, Dict[str, Any]]): Analysis results Returns: Dict[str, Any]: Summary statistics r)total_documents categoriesavg_confidencerz Other:UnknownrrcSs$g|]}|dddkr|dqS)rrg333333?r)r)rpresultririrjrs7s z9DocumentAnalyzer.get_analysis_summary..)rrrlow_confidence_docs)valuesrappendsumr)rhrr confidencesrrrrririrjget_analysis_summarys&     z%DocumentAnalyzer.get_analysis_summaryN) __name__ __module__ __qualname____doc__rrkrvrrrrrriririrjrs67*g.rrrlcCst|S)z Factory function to create a DocumentAnalyzer instance. Args: claude_api_key (str): API key for Claude Returns: DocumentAnalyzer: Initialized analyzer instance )r)rririrjcreate_document_analyzer?s r)rrloggingrtypingrrrrrcrdotenvrgetenvrr r r r rfr rb getLoggerrrrrrriririrjs*          "