Expression of reporter genes under the control of the HIV-1 long terminal repeat (LTR) was up-regulated in the murine macrophage cell line RAW264 by cotransfection of a plasmid coding for the viral regulatory protein Nef. To determine if a discrete section of the LTR was exclusively responsive to Nef, a series of promoters was produced by successive 5' deletions from the LTR up to the boundary of the enhancer region. These truncated promoters were as active as the full-length sequence in the RAW264 cells, but elimination of the direct repeats and one of the three Sp1 sites reduced promoter activity to minimal levels. Transcription driven by all constructs was equally susceptible to the trans-activating effect of Nef and could be increased further by the addition of a Tat-expressing plasmid to the cotransfection. Open reading frames of nef from NL4-3, from HXB2, which has a premature stop, and a fully functional hybrid of the two under the control of the SR alpha artificial promoter (SV40 early promoter plus HTLV-I R-U5') were able to transactivate the LTR in RAW264 cells to the same degree as HXB3 nef under the control of the cytomegalovirus (CMV) immediate-early promoter. A frameshift mutation of Nef at the XhoI site at position 8475 did not abrogate trans-activation of the LTR in macrophages. To further define the effective trans-activation region of Nef, internal deletions were made. Changes downstream of the XhoI site at amino acid 35 resulted in little or no reduction in trans-activation, whereas a deletion between the CMV promoter of the expression plasmid and the XhoI site largely abolished activity. Nef trans-activation of the LTR may be restricted to macrophages. Parallel cotransfection experiments in COS-1 simian fibroblast-like cells showed repression of reporter expression by Nef. Results suggested that the section of nef responsible for transactivation of the LTR in macrophages differed slightly from that sufficient for trans-repression in fibroblasts. Translation of the protein from the first translation start site (Met-1) rather than from the second in-frame ATG (Met-20) appears to be necessary for the trans-activating effect of Nef in RAW264 cells. Mutation of the initial ATG to ATA led to loss of trans-activating activity. Expression of Nef also has a cytostatic/cytotoxic effect on RAW264 cells indicated by a reduced rate of establishment of stably transfected clones. The cytostatic effect of Nef was not relieved by internal deletions in the coding sequence.(ABSTRACT TRUNCATED AT 400 WORDS)